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Abstract 

To  exploit  large  deviation  approximations  for  allocation  and  occupancy 
problems  one  must  solve  a  deterministic  optimal  control  problem  (or  equiv¬ 
alently,  a  calculus  of  variations  problem).  As  this  paper  demonstrates,  and  in 
sharp  contrast  to  the  great  majority  of  large  deviation  problems  for  processes 
with  state  dependence,  for  allocation  problems  one  can  construct  more-or-less 
explicit  solutions.  Two  classes  of  allocation  problems  are  studied.  The  first 
class  considers  objects  of  a  single  type  with  a  parameterized  family  of  placement 
probabilities.  The  second  class  considers  only  equally  likely  placement  proba¬ 
bilities,  but  allows  for  more  than  one  type  of  object.  In  both  cases,  we  identify 
the  Hamilton- Jacobi-Bellman  equation  whose  solution  characterizes  the  mini¬ 
mal  cost,  explicitly  construct  solutions,  and  identify  the  minimizing  trajectories. 
The  explicit  construction  is  possible  because  of  the  very  tractable  properties  of 
the  relative  entropy  function  with  respect  to  optimization. 


1  Introduction 

Allocation  and  occupancy  problems  are  concerned  with  the  random  placement  of 
objects  into  containers.  The  objects  (usually  referred  to  as  balls  or  tokens)  can  be 
of  a  single  type  or  many,  in  which  case  they  are  often  distinguished  by  “color.”  The 
containers  are  variously  called  urns  or  cells,  and  have  many  interpretations,  such  as 
physical  partitions  (photo-electric  receptors  in  a  grid)  and  temporal  partitions  (the 
days  of  the  year). 

There  are  also  many  rules  for  how  a  given  ball  may  be  assigned  to  a  given 
cell.  The  simplest  such  rule,  in  the  context  of  a  single  color,  uses  what  are  called 
Maxwell-Boltzmann  (MB)  statistics.  Here,  each  cell  is  equally  likely  to  receive  each 
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ball.  Other  rules  consider  the  balls  as  being  placed  sequentially,  and  the  likelihood 
that  a  given  ball  is  placed  in  a  given  cell  depends  on  the  current  contents  of  that 
cell  (relative  to  the  contents  of  all  other  cells).  Examples  in  this  category  are  Bose- 
Einstein  statistics  (BE),  for  which  a  cell  that  already  contains  balls  is  more  likely  to 
receive  the  next  ball,  and  Fermi-Dirac  statistics  (ED),  where  the  reverse  holds.  The 
precise  definitions  of  BE  and  ED  will  be  given  below. 

A  key  random  variable  associated  with  an  allocation  is  the  empirical  measure. 
After  all  (or  some)  of  the  balls  have  been  placed,  one  can  form  the  (random)  proba¬ 
bility  measure  {rjo,  771, . . .)  on  {0, 1, . . .},  with  rjo  equal  to  the  fraction  of  cells  that  are 
empty,  rji  the  fraction  that  contain  1,  etc.  For  example,  one  could  be  particularly 
concerned  that  at  least  90%  of  the  cells  are  nonempty  after  the  random  allocation.  In 
this  case  one  is  interested  in  the  distribution  of  the  first  component  of  the  empirical 
measure,  and  in  particular  P{r]Q  <  0.1}. 

While  methods  from  combinatorial  probability  provide  exact  formulas  for  certain 
classes  of  allocation  problems,  they  do  not  apply  universally,  nor  are  they  always 
of  great  practical  utility-see  the  discussion  in  [2]  on  this  point.  Hence  one  turns 
to  approximations.  The  simplest  approximation  is  a  law  of  large  numbers  (LLN) 
limit,  under  which  the  number  of  cells  and  number  of  balls  placed  into  the  cells  both 
tend  to  00  with  some  fixed  ratio.  If  i]  is  indexed  by  the  number  of  cells  n,  then  the 
LLN  limit  identifies  the  (deterministic)  probability  distribution  that  ry”  tends  to  as 
n  — >  00.  This  identifies  the  “typical”  behavior  of  the  allocation  scheme  for  large  n. 
The  limit  can  often  be  identified  as  the  solution  to  a  system  of  ordinary  differential 
equations  (ODEs)  at  time  t  (and  for  an  appropriate  initial  condition),  where  t  is 
limiting  ratio  of  the  number  of  balls  to  the  number  of  cells,  i.e.,  the  mean  number 
of  balls  per  cell. 

If  in  contrast  one  is  concerned  with  probabilities  of  atypical  behavior,  then  one 
considers  large  deviation  asymptotics.  For  example,  if  it  is  usual  that  50%  of  the  cells 
are  empty  when  n  is  large,  then  under  some  technical  assumptions  large  deviation 
asymptotics  assert  that  —^logP{rio  <  0.1}  tends  to  some  constant  c  >  0,  thus 
identifying  the  exponential  rate  of  decay  of  the  probability.  The  parameter  c  is 
usually  identified  as  the  solution  of  a  calculus  of  variations  problem,  and  using  the 
well  known  relation  between  problems  in  calculus  of  variations  and  Hamilton- Jacobi 
equations,  c  can  also  be  characterized  as  the  value  (at  a  particular  point)  of  the 
solution  to  a  nonlinear  partial  differential  equation  (PDE). 

The  explicit  identification  of  c  is  in  general  a  daunting  task.  Whilst  there  are 
a  small  number  of  cases  for  which  analytic  expressions  are  available,  in  most  cases 
one  must  attempt  numerical  approximation,  and  so  one  is  limited  to  only  low  di¬ 
mensional  problems  (i.e.,  in  our  setting  to  the  first  few  components  of  the  empirical 
distribution).  Even  putting  aside  the  restriction  of  numerical  methods  to  low  di¬ 
mensions,  one  would  prefer  analytic  expressions  for  c  since  they  have  many  other 
uses.  Beyond  simply  identifying  the  rate  of  decay,  analytic  expressions  for  c  can  be 
used 


•  to  characterize  the  most  likely  way  that  a  rare  event  will  occur, 

•  to  construct  efficient  Monte  Carlo  schemes  (known  as  importance  sampling 
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schemes)  for  non-asymptotic  approximations,  and 

•  in  statistical  estimation  and  model  inference  for  occupancy  models. 

The  purpose  of  the  present  paper  is  to  show  that  explicit  solutions  can  be  ob¬ 
tained  for  the  PDEs  that  are  associated  with  a  wide  variety  of  allocation  problems, 
and  introduce  techniques  that  can  be  applied  to  even  broader  classes  of  problems. 
As  remarked  previously,  explicit  solutions  are  not  common.  Among  the  classes  of 
nonlinear,  first  order  PDE  with  explicit  solutions  (in  general  dimension)  are  those 
associated  with  the  linear  quadratic  regulator  and  those  linked  to  the  Hopf-Lax  for¬ 
mula.  Both  these  examples  exploit  some  significant  underlying  simplification.  In 
the  first  example  it  is  the  fact  that  the  value  function  for  the  control  problem  is 
expected  to  be  quadratic  in  the  spatial  variable,  and  in  the  second  example  it  is  the 
independence  of  the  running  cost  from  the  state  variable.  The  optimization  prob¬ 
lems  related  to  allocation  problems  are  qualitatively  quite  different  from  either  of 
these,  as  can  be  seen  from  both  the  form  of  the  value  functions  and  the  structure  of 
the  minimizing  trajectories.  There  is  significant  state  dependence,  and  no  a  priori 
obvious  form  for  the  value  function.  In  the  setting  of  allocation  problems,  it  seems 
that  the  attractive  properties  of  the  relative  entropy  function  are  largely  responsible 
for  the  existence  of  explicit  solutions.  It  is  these  properties  which  allow  for  conve¬ 
nient  calculation  and  representation  of  the  various  derivatives  in  terms  of  Lagrange 
multipliers,  the  key  ingedient  in  the  proof. 

In  the  next  section  we  analyze  the  single  color  model.  After  introducing  the 
general  model  and  formally  reviewing  the  large  deviation  context,  we  discuss  a  formal 
and  heuristic  derivation  of  the  explicit  solution.  The  associated  Hamilton- Jacobi- 
Bellman  (HJB)  equation  is  then  introduced,  and  a  solution  is  proposed  in  the  form  of 
a  finite  dimensional  minimization  problem  that  can  be  easily  and  efficiently  solved 
using  Lagrange  multiplier  techniques.  The  value  of  the  minimization  problem  is 
shown  to  be  smooth  for  an  appropriate  class  of  terminal  costs,  its  derivatives  are 
characterized  via  multipliers,  and  the  HJB  equation  is  shown  to  hold.  The  section 
concludes  with  the  identification  of  the  minimizing  trajectories.  The  third  and  final 
section  repeats  these  steps  for  a  model  with  different  colors. 

2  Allocation  Models  with  Differing  Assignment  Proba¬ 
bilities 

2.1  Probabilistic  Background  and  the  Variational  Problem 

In  this  section,  we  formulate  a  general  single  color  occupancy  problem.  After  de¬ 
scribing  the  model,  we  outline  the  relevant  large  deviation  properties  on  path  space 
and  the  related  variational  problems. 

In  the  occupancy  problem  considered  here  cells  are  distinguished  according  to 
the  number  of  balls  contained  therein.  The  full  collection  of  models  will  be  indexed 
by  a  parameter  a.  This  parameter  takes  values  in  the  set  (0,  oo]  U  {  — 1,  —2, . . and 
its  interpretation  is  as  follows.  Suppose  that  a  ball  is  about  to  be  thrown,  and  that 
any  two  cells  (labeled  say  A  and  B)  are  selected.  An  cell  is  said  to  be  of  category  i 


3 


if  it  contains  i  balls.  Snppose  that  cell  A  is  of  category  i,  while  B  is  of  category  j. 
Then  the  probability  that  the  ball  is  thrown  into  cell  A,  conditioned  on  the  state  of 
all  the  cells  and  that  the  ball  is  thrown  into  either  cell  A  or  S,  is 


(a  +  i)  +  {a  +  j)' 

When  a  =  oo  we  interpret  this  to  mean  that  the  two  cells  are  eqnally  likely.  Also, 
when  o  <  0  we  use  this  ratio  to  define  the  probabilities  only  when  0  <  i  V  j  < 
—a  and  i  <  —  a  or  j  <  —a,  so  the  formula  gives  a  well  defined  probability.  The 
probability  that  a  ball  is  placed  in  an  cell  of  category  —a  is  0.  Thus  under  this  model, 
cells  can  only  be  of  category  0, 1, ... ,  —a,  and  we  only  throw  balls  into  categories 
0, 1, ... ,  —a  —  1. 

When  a  G  (0,  oo)  cells  that  already  contain  balls  are  more  likely  to  receive  the 
next  ball.  When  a  <  0  the  opposite  is  true.  The  cases  o  =  l,a  =  oo,aG 
— N  correspond  to  what  were  called  Bose-Einstein  statistics,  Maxwell-Boltzmann 
statistics,  and  Fermi-Dirac  statistics,  respectively,  in  the  Introduction. 

Suppose  that  before  we  throw  a  ball  there  are  already  tn  balls  in  all  the  cells, 
and  that  the  occupancy  state  is  (rco,  xi, . . .  xi+).  Here  Xj,  i  =  0, 1, . . . ,  /  denotes  the 
fraction  of  cells  that  contain  i  balls,  and  X7+  denotes  the  fraction  containing  more 
than  I  balls.  Throughout  this  paper  we  use  this  convention  so  that  the  state  space 
of  the  occupancy  process  is  finite  dimensional.  (Explicit  formulas  analogous  to  the 
ones  derived  here  also  hold  in  the  infinite  dimensional  case,  though  one  must  be 
more  careful  in  defining  the  PDE.)  When  the  occupancy  state  is  (xq,  xi, . . .  x/+), 
the  “un-normalized”  or  “relative”  probability  of  throwing  into  a  category  i  cell  with 
i  <  /  is  simply  (o  +  i)xi.  Let  us  temporarily  abuse  notation,  and  let  x/+i,X7+2, . . . 
denote  the  exact  fraction  in  each  category  i  with  i  >  I.  Since  there  are  tn  balls  in 
the  cells  before  we  throw,  ~  Thus  the  (normalized  and  true)  probability 

that  the  ball  is  placed  in  an  cell  that  contains  exactly  i  balls,  i  =  0, 1, ...  I,  is 
and  the  probability  that  the  ball  is  placed  in  an  cell  that  has  more  than  I  balls  is 

In  order  to  define  both  the  LLN  and  large  deviation  approximations,  it  is  conve¬ 
nient  to  introduce  an  occupancy  process.  We  introduce  a  time  variable  t  that  ranges 
from  0  to  T.  At  a  time  t  that  is  of  the  form  l/n,  with  0  <  I  <  [nTj  an  integer,  I  balls 
have  been  thrown.  Let  X^{t)  =  {  Aq  (f),  Af'(f), . . .  be  the  occupancy 

state  at  that  time.  As  noted  previously,  Xf{t)  denotes  the  fraction  of  cells  that 
contain  i  balls  at  time  f,  f  =  0, 1, . . .  /,  and  A]^_(t)  the  fraction  of  cells  that  contain 
more  than  I  balls.  The  definition  of  A”  is  extended  to  all  t  G  [0,  T]  not  of  the  form 
l/nhy  piecewise  linear  interpolation.  Note  that  A"'(t)  is  indeed  a  probability  vector 
in  M-f+2.  If 

Si  =  |x  G  :xi>0, 0<z</  +  l  and  ^  Xj  =  l|  , 

then  for  any  t  G  [0,  T]  ,  A"'(t)  G  Si.  Thus  A”  takes  values  inU  =  C  ([0,  T]  ,  Si).  We 
equip  U  with  the  usual  supremum  norm  and  on  Si  we  take  the  usual  Li  norm. 
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It  is  often  the  case  that  one  is  interested  in  the  large  deviation  properties  at 
the  terminal  time  T  (i.e.,  those  of  X^{T)),  and  for  a  general  initial  condition  of  the 
form  X^{t)  =  {xq,  . . . ,  xi+).  Here  there  is  often  a  detour-one  first  identifies  the  large 
deviation  properties  of  the  process,  and  then  solves  for  the  large  deviation  properties 
of  X"'(r)  via  the  so-called  Contraction  Mapping  Theorem.  This  theorem  represents 
the  sought  after  exponential  rate  of  decay  as  the  solution  to  a  calculus  of  variations 
problem,  and  therein  lies  the  link  to  a  PDE. 

For  our  purposes  an  informal  description  of  the  process  level  large  deviation 
properties  will  sufRce.  We  first  define  the  rate  function  on  path  space.  Given  {x,  t) 
and  a  continuous  trajectory  (p  with  ip{t)  =  x,  the  rate  x,  t)  identifies  the  decay 
rate  for  the  probability  that  X'^  is  in  a  small  neighborhood  of  p: 


lim  lim  sup - log  P 

n — >oo  n 


sup  |d^"'(s)  —  (/9(s)|  <  6 

t<s<T 


X^{t) 


=  X 


=  lim  lim  inf - log  P 

fij,0  n^oo  n 


sup  |^”(s)  —  97(5)1  <  6 

t<s<T 


X^{t) 


=  X 


=  l{ip;x,f). 


Here  x^  is  any  sequence  of  initial  conditions  that  can  occur  with  positive  probability 
and  which  satisfy  x""  — >  x  as  n  — 00.  The  proof  of  such  a  result  and  the  identification 
of  the  rate  function  are  given  in  [7].  P{p;  x,  t)  can  be  represented  as  the  integral,  over 
[t,T],  of  a  non- negative  “cost”  which  measures  the  likelihood  that  the  increments  of 
X^  follow  the  increments  of  ip,  with  higher  cost  corresponding  to  lower  likelihood 
(the  LLN  trajectory  has  zero  cost).  The  integral  form  of  P{p;  x,  t)  is  a  consequence 
of  the  Markov  property. 

The  specific  form  of  P{p>]  x,  t)  is  as  follows.  Define  the  linear  map  M  :  Sj  i—>- 

r  -00  i  =  0 

Mi[9]  =  I  l<i<I  . 

[  9i  i  =  I  +  l 

Let  pi  G  U  he  given  with  p{t)  =  x.  Suppose  there  is  a  Borel  measurable  function 
9  :  [t,  T]  I — >  Si  such  that  for  any  s  G  [t,  T] 

p{s)  =  p{t)  +  j  M[9]{u)du.  (2-1) 

We  interpret  9i{s)  as  the  rate  at  which  balls  are  thrown  into  cells  that  contain  i 
balls  at  time  s.  This  rate  will  be  viewed  as  a  perturbation  of  the  LLN  limit  rate 
at  which  balls  would  be  thrown,  and  the  cost  for  this  perturbation  will  measure  the 
likelihood  that  sure  a  perturbation  occurs  (with  large  cost  corresponding  to  unlikely 
perturbations). 

The  mapping  M[9\  accounts  for  the  fact  that  when  a  ball  is  placed  in  a  category 
i  cell  Xf  decreases  by  1/n  and  Xff^^  increases  by  1/n.  The  rates  9{s)  are  unique  in 
the  sense  that  if  another  9  :  [t,  T]  \ — >  Si  satisfies  (2.1)  then  9  =  9  a.e.  on  [t,  T] .  We 
call  p  a  valid  occupancy  state  process  if  there  exists  9  :  [t,  T]  1 — >  Si  satisfying  (2.1). 
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In  this  case  6  is  called  the  occupancy  rate  process  associated  with  ip.  For  x  G 
and  t  G  [0,  — al{a<o}  +  ool{a>o}))  define  the  vector  p{t,x)  G  by 


Pk{t,x) 


a  +  k 
a  +  t 


for  A:  =  0, 1, ...  I, 


(2.2) 


and 


,  ,  V  ^  d  -\-  Jv 

Pl+[t,x)  =  1  -  >  - —Xk- 

a  +  t 

k=0 

For  each  a,  pk{t,x)  gives  the  LLN  limiting  probability  that  at  time  t  the  next  ball 
will  be  placed  in  a  category-A:  cell,  given  that  the  statistics  of  model  a  are  used  and 
that  X"'{t)  =  X.  A  direct  calculation  shows  that  if 

/+i 

X  G  Sj  and  kxk  <  t,  (2-3) 

k=0 

then  p{t,x)  is  indeed  a  probability  vector  in  i.e.,  p{t,x)  G  Sj.  It  is  easy  to 

observe  that  if  p  is  valid  then  p{s)  satisfies  (2.3)  for  all  s  G  [0,  T].  This  shows  that 
p  (s,  (/?(s))  G  Si-  For  future  use  we  define 


t{x,  t)=  “  X]  j  j 

if  X7+  >  0  and  r(x,  A)  =  I  +  I  if  xjj^  =  0.  Thus  r(x,  t)  can  be  interpreted  as  the 
mean  number  of  balls  per  cell  among  those  of  category  /+.  With  this  notation 

pi+{x,  t)  =  {a  +  t{x,  t))xi+i/ (o  +  A),  (2.5) 

and  so  pi+{x,t)  in  some  sense  takes  a  form  very  similar  to  that  of  pk{x,t)  for 
A:  =  0, 1, . . . ,  I. 

Let  (5  >  0  be  small.  Observe  that  the  occupancy  state  will  not  change  very  much 
over  [A,  A  +  (5]  while  n6  balls  are  placed  into  cells.  Let  9  denote  the  empirical  measure 
on  the  categories  where  these  balls  are  placed.  Then  the  new  occupancy  state  is 
the  sum  of  the  old  state  plus  6M[9].  Since  the  change  in  state  is  determined  by 
an  empirical  distribution  for  (at  least  approximately)  iid  random  variables,  Sanov’s 
Theorem  [1,  Theorem  2.2.1]  suggests  that  the  cost  appearing  in  the  integral  rep¬ 
resentation  for  I{p',  X,  A)  should  be  defined  in  terms  of  the  famous  relative  entropy 
function.  For  two  probability  measures  a  and  /3  on  a  Polish  space  A,  the  relative 
entropy  of  a  with  respect  to  (3  is  defined  by 

Riam  =  l^(log+yn 

whenever  a  is  absolutely  continuous  with  respect  to  (5  (and  with  the  convention  that 
OlogO  =  0).  In  all  other  cases  we  set  R{a\\(5)  =  oo.  When  two  probability  vectors  p 
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and  V  ^  Sj  appear  in  the  relative  entropy  fnnction,  we  interpret  them  as  probability 
measures  on  the  simplex  {0, 1, I  +  1},  and  thns 

i+i 

•  A 
2=0 

Important  properties  of  relative  entropy  are  that  it  is  nonnegative,  jointly  convex 
and  lower  semicontinuous  in  (a,  (3),  and  R  (a||/3)  =  0  if  and  only  \i  a  =  (3  [1,  Lemma 
1.4.3]. 

As  observed  before,  when  ip{s)  is  valid,  p  (s,  (p(s))  G  Sj,  which  makes  R{9{s)\\p  (s,  (p(s))) 
well  dehned.  If  in  addition  (p(t)  =  x,  define 

I{(p-,x,t)  =  1^  R{e{s)\\p{s,(p{s)))ds.  (2.6) 

If  (p  is  not  valid  or  (p{t)  ^  x  then  dehne  Kpp\  x,  t)  =  oo. 

This  defines  the  rate  fnnction  for  the  models  introduced  at  the  beginning  of 
this  section.  Now  suppose  that  one  wishes  to  approximate  probabilities  involving 
X"'{T).  Since  the  probability  that  A”  (as  a  process)  is  close  to  a  given  trajectory  ip 
decays  exponentially,  decay  rates  of  quantities  such  as  P{X^{T)  G  A\  X^{t)  =  x"'} 
can  (under  appropriate  regularity  conditions  on  A)  be  fonnd  as  follows.  Among  all 
trajectories  p  with  p{t)  =  x  and  p{T)  G  A,  identify  the  one  with  the  smallest  decay 
rate  c.  Then  c  is  also  the  exponential  decay  rate  of  P{X^{T)  G  A\  X^{t)  =  x"'}. 

Hence  the  variational  problem  to  be  solved  is 

V{x,t)=  inf  I{p;x,t).  (2.7) 

ip:(f(t)=x  and  ip(T)£A 

If  one  is  interested  in  expected  values  other  than  probabilities  then  variational  prob¬ 
lems  of  the  more  general  form 

V{x,t)=  inf  [I{p;x,t)  +  F{p{T))]  (2.8) 

(f:(p{t)=x 

arise,  and  one  is  often  particularly  interested  in  the  initial  condition  that  corresponds 
to  starting  with  all  cells  empty:  t  =  0,  xq  =  1  and  x^  =  0,  k  >  0.  We  will  refer  to 
this  as  the  empty  initial  condition. 

Not  all  initial  conditions  are  feasible,  in  the  sense  that  they  can  be  reached  with 
finite  cost  from  the  empty  initial  condition.  Feasibility  in  this  context  depends  on 
the  nnderlying  parameter  a. 

Definition  2.1  (Feasible  Domain).  Define  Da,  the  feasible  domain  for  the  oceu- 
pancy  model  with  parameter  a,  as  follows: 

•  when  a  >  0, 
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•  and  when  a  <  0  and  I  =  —a  —  1 


Va^  <  {x,  t)  e  Si  X  [0,T)  :  t  =  ixi 
I  i=0 

In  the  first  case  the  second  set  in  the  union  reflects  the  fact  that  when  xj+i  =  0 
the  number  of  balls  thrown  is  exactly  similarly  for  the  second  case. 

When  a  G  — N  it  is  only  possible  to  throw  balls  into  the  categories  0, 1, . . . ,  — o  —  1, 
and  the  only  possible  categories  are  0, 1, ,  —a.  Thus  if  there  are  n  cells  there  can 
at  most  be  —an  balls  thrown,  and  therefore  T  <  —a.  When  T  =  —a  all  the  cells 
have  exactly  —a  balls,  which  is  not  an  interesting  case  to  study.  As  a  consequence, 
throughout  this  paper  we  assume  T  <  —a.  Also,  because  of  the  restriction  on  the 
possible  categories  we  can  (without  loss)  assume  that  /  =  —  a  —  1.  Hence  for  o  <  0 
we  assume  without  loss  that 


T<-a,  I  =  -a-l.  (2.9) 

2.2  LLN  Limits  and  Formal  Derivation  of  the  Explicit  Solntion 

When  constructing  explicit  solutions  one  needs  some  insight  into  the  form  of  the 
solution.  In  this  section  we  present  a  formal  derivation  of  an  explicit  solution  to 
(2.8)  for  the  case  F{x)  =  ly{x)  ■  oo.  Before  doing  so  we  calculate  the  LLN  limits  of 
the  occupancy  processes,  a  necessary  ingredient  in  the  solution. 

Equations  for  the  LLN  limits  can  easily  be  derived  directly,  or  alternatively  by 
noting  that  they  are  the  zero  cost  trajectories  in  the  variational  problem  (2.8)  with 
F  =  0.  It  will  suffice  to  consider  initial  conditions  of  the  form  x  =  e^,.  A:  =  0, 1, . . . ,  I, 
where  {ek)j  is  I  if  j  =  k  and  zero  otherwise.  Since  the  relative  entropy  vanishes  only 
if  0(s)  =  p  (s,  <p(s)),  the  LLN  limits  can  be  characterized  by  the  system  of  ODEs 

99(5)  =  M[p(s,(/9(s))],  ip{t)  =  ek.  (2.10) 

Since  the  LLN  limit  is  desired  for  all  components  of  the  occupancy  process,  we  use 
the  infinite  system  rather  than  the  system  truncated  at  /+.  These  are  easy  to  solve 
because  the  equation  for  the  jth  component  depends  only  on  the  j  —  1st  component, 
and  so  one  can  solve  first  for  the  feth  component  and  then  bootstrap.  To  write  the 
solution  in  explicit  form,  we  need  some  notation.  For  all  a  G  M,  a  7^  0  and  f  G  N,  let 

fa\  ^  UyJoia-j) 

\V 

Note  that  if  a  G  N  and  i  >  a  then  (“)  =  0,  and  that  if  a  ^  N  U  {0}  and  i  G  N,  then 
(“)  7^  0.  For  i  G  N  U  {0}  and  o>0,  s>0oraG  — N,  0  <  s  <  —a,  define 


One  can  easily  check  that  the  solution  to  (2.10)  is  (pi{s)  =  0  if  i  <  k,  and 
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\i  k>i.  In  the  limit  a  — >  oo  (MB  statistics)  one  obtains  the  Poisson  distribution 

(^(8  -  *))  ^  Vi(,  -t)  =  -  tyni 

For  the  remainder  of  this  section  we  assume  a  ^  oo,  with  the  understanding  that 
analogous  statements  for  o  =  oo  can  be  obtained  by  passing  to  the  limit. 

We  next  present  a  formal  and  heuristic  solution  to  the  variational  problem  based 
on  probabilistic  intuition.  Recall  that  the  variational  problem  is  intended  to  approx¬ 
imate  the  normalized  logarithm  of  a  probability.  If  one  decomposes  a  probability 
into  products  or  conditional  products,  this  will  correspond  to  a  decomposition  of 
the  quantity  being  minimized  as  a  sum. 

We  wish  to  solve  (2.7)  when  A  =  {y}.  Suppose  that  Xi  is  interpreted  as  the 
size  of  the  “pool”  of  cells  that  start  at  time  t  in  category  i.  Through  the  random 
placements,  this  pool  will  evolve  into  sub-pools  of  differing  categories.  Let  denote 
the  probability  that  a  cell  of  category  k  at  time  t  ends  up  a  cell  of  category  k  +  i  at 
time  T.  Then  satisfaction  of  the  terminal  constraint  requires 

i  I 

yi  =  ^ XkTTi-k,  0  <  f  <  /,  yi+i  =  1  -  ^  yfe. 
k=0  k=0 

We  use  y  =  X  X  TT  as  shorthand  for  the  last  display.  We  require  that  the  be 
probabilities,  and  also  a  constraint  that  corresponds  to  the  fact  that  n(T  —  t)  balls 
will  be  placed  in  the  prelimit  problem: 

OO  7+1  OO 

Xk^Trj=Xk,  0<k<I  +  l,  ^Xk^jTTj  =T -t,  (2.11) 

j=0  k=0  j=0 

Let  !F{x,t',y,T)  denote  the  set  of  tt  =  (+,  vr^, ...,+,  vr^^^)  which  satisfy  the  last 
two  displays.  A  terminal  point  y  is  feasible  (for  the  given  initial  time  and  condition) 
if  iF{x,  t]  y,  T)  is  not  empty. 

To  guess  the  form  of  the  solution  to  the  variational  problem,  we  consider  the 
allocation  from  a  different  perspective.  Owing  to  the  fact  that  the  un-normalized 
relative  probabilities  are  affine  in  the  number  of  balls  currently  in  each  cell,  we 
can  first  study  the  random  evolution  of  the  number  of  balls  that  are  in  each  pool. 
This  can  be  done  without  knowing  the  details  of  how  the  balls  are  placed  within 
the  pool.  Indeed,  the  structural  properties  of  the  placement  probabilities  imply  that 
this  process  is  also  Markovian,  and  its  large  deviation  properties  are  easy  to  identify. 

Once  we  know  the  total  number  of  balls  that  will  end  up  in  each  pool,  we  then 
consider  the  question  of  how  they  are  distributed  among  cells  within  the  pool.  Here 
we  make  an  approximation  that  is  formal  but  reasonable,  which  is  that  the  rate 
function  for  the  empirical  distribution  within  the  pool  can  be  found  as  follows.  We 
first  generate  iid  random  variables  according  to  the  LLN  distribution  appropriate 
to  the  particular  pool,  and  form  the  empirical  distribution  for  this  sample.  The 
rate  function  for  this  empirical  distribution  is  a  certain  relative  entropy  identified  by 
Sanov’s  Theorem.  However,  we  must  also  impose  the  constraint  on  the  (previously 
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determined)  number  of  balls  that  were  placed  into  this  particular  pool,  which  adds 
a  constraint  to  the  rate  function.  Finally,  the  overall  rate  function  is  found  by 
combining  these  two  rates.  The  function  found  by  this  manner  will  be  proved  to  be 
the  solution  to  the  calculus  of  variations  problem. 

An  argument  based  on  Sanov’s  Theorem  shows  that  the  variational  problem  for 
the  allocation  between  the  pools  is 


inf 


R  {u{s)  ||rc(s) )  ds, 


where 

{a  +  k)xk  +  J^Uk{T)dT 
a  +  s 

is  the  probability  that  a  ball  is  placed  into  pool  k  at  time  s.  This  is,  in  un-normalized 
form,  equal  to  axfc+[number  of  balls  per  cell  in  pool  k]xk,  and  the  normalization  is 
just  a  +  s.  The  initial  and  terminal  conditions  are 


Wk{s) 


Wk{t) 


(o  +  k)xk 
a  +  t 


Wk{T) 


(g  +  k)xk  +  Zk{T  -  t) 
a  +  T 


where  Zk  is  the  mean  number  of  additional  balls  per  unit  time  put  into  pool  k.  The 
Euler-Lagrange  equations  for  this  problem  are  easily  constructed  and  solved,  and 
one  obtains  as  the  optimal  trajectory 

Wk{s)  =  ((a  +  k)xk  +  (s  -  t)zk) 

s  +  a 


(of  course  satisfaction  of  the  Euler-Lagrange  equations  is  not  in  general  a  sufficient 
condition  for  optimality,  but  since  our  discussion  is  simply  to  motivate  the  form  of 
the  solution  this  point  is  of  no  consequence).  The  cost  is 


R  ([(s  -I-  o)rc(s)]'  ||rc(s))  ds, 


and  for  the  optimal  trajectory 


[(s  +  o)Wfc(s)]'  =  Zk. 


The  integral  can  be  explicitly  evaluated,  and  equals 


/+ 

'^Zk-  {T  -t)  ■  log 


k=0 


k  a  +  t 
\a  -|-  k 


1+ 

-'^[xk  ■  {a  +  k)  +  Zk  ■  (T  -  t)]  ■  log 


k=0 


/„  +  t+a(r-t)\ 

[a  +  k  +  <S^{T-t))^ 


This  identifies  the  first  part  of  the  overall  rate  function. 

The  second  part  is  found  by  considering  placement  within  each  pool.  The  mean 
additional  number  of  balls  per  cell  in  pool  k  is  {zk/xk){T  —  t).  According  to  the 
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LLN,  the  number  of  additional  balls  in  a  typical  cell  from  pool  k  has  distribution 


a+k 


Zk  (rji 

Xk  '' 


t)]  if  k  <  I  and  Q 


a+T{x,t)  (  /rp  _ 


\^I+ 


{T  —  t)^  if  k  =  I+.  Approximating 


Q' 

the  true  empirical  measure  within  a  given  pool  by  that  of  the  empirical  measure  for 
iid  random  variables  with  the  corresponding  distribution,  one  formally  obtains  from 
Sanov’s  theorem  the  rate  function  R 

constraint  ~  numt 

the  different  contributions  from  the  various  pools  with  the  contribution  due  to  the 
allocation  between  the  pools  and  then  applying  the  terminal  constraint,  one  (again 
formally)  obtains  the  rate  function 


— t)j  j,  together  with  the 
ber  of  balls  placed  in  pool  k.  Combining 


inf 


XkR  TT 


Q- 


a-\-k 


-(T-t) 

Xk 


+XI+R  ( 7r^+  (^{T-  t) 

\  \xi+ 


1+ 

E 

fc=0 


[xk  ■  {a  +  k)  +  Zk- {T  -  t)]  •  log 


\ 

j  +  ^Zk-  {T  -t)  -  log 

'  k=0 

a  +  k+^(T-t) 


a  +  t 
a  +  k 


£fe 

Xk 


where  the  infimum  is  over  all  vr  and  z  such  that  ~  xxir  =  y. 

However,  a  straightforward  calculation  using  the  specific  form  of  and  '^'^i  ~ 


^{T-t)  gives 


R  I  vr 


Q' 


a+k 


a  +  k 
a  +  t 


Xk 


(T-t)-  log 


{T-t) 


a  +  t 


—  R  [  TT 


a+k 


a  +  k 
{zk/xk){T  -t)  -  log 


ffc 

Xk 


Q 


-  {a  +  k)  ■  log 


-{T-t) 

^Xk 

'  a  +  k++^{T-t) 

a  +  k  +  ^(T-t) 


a  +  k+s^(T-t) 
a  +  k  +  ^(T-t)_ 


with  an  analogous  result  for  k  =  I+.  If  follows  that  the  rate  function  can  be  written 
in  the  simpler  form 


inf 

7r:xX7r=y 


XkR  vr 


Q' 


a+k 


a  +  k 
a  +  t 


{T-t) 


+  Xl+R  TT 


R+ 


Qa+T{x,t)  f  a  +  T{x,t)  _ 

\  a  + 1 


with  the  infimum  over  z  no  longer  necessary. 
Let 


J  {x,t;y)  = 


inf  T{xA\ip). 
v.eC([t,T],5H 
ip{t)=x,tp{T)=y 


(2.12) 


The  formal  derivation  just  given  suggests  the  following  result,  in  which  we  also 
simplify  further  where  the  special  cases  of  FD  and  MB  statistics  allow. 
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Theorem  2.2  (Explicit  Formula  for  the  Rate  Function).  Consider  an  initial 
condition  {x,t)  G  Va,  and  a  feasible  terminal  condition  y.  If  a  G  (0,oo),  define 
T{x,t)  by  (2.4)-  Then  the  quantity  J  {x,t;y)  defined  in  (2.12)  has  the  representation 

+  Xl+lR  (^TT^+l  Q-+ri-,t)  -  t))  )  }  . 

If  a  G  — N  with  I  =  —a  —  1  then  t{x,  t)  =  I  +  1,  and 

In  the  final  case  of  a  =  00,  we  have 


J  {x,t;y)  =  min 

'K&!F{x,t-,y,T) 


J  {x,  t]  y)  =  min 

■K&T(x,t\y,T) 


'^XkR 


J{x,t-,y)=  min 


J2^kRU'^\\V{T- 


Remark  2.3.  Although  the  minimization  problems  in  Theorem  2.2  appear  to  be 
infinite  dimensional,  they  can  in  fact  be  reduced  to  finite  dimensional  problems. 
This  is  because  if  is  the  minimizer,  then  vr^  takes  a  prescribed  form  for  j  >  I.  In 
fact,  all  vrj^  can  be  represented  in  terms  of  no  more  than  /  +  3  Lagrange  multipliers 
as  in  (2.20)  below. 


2.2.1  The  Hamilton-Jacobi-Bellman  equation 

Given  Theorem  2.2  one  can  solve  the  problem  with  a  general  terminal  condition  F. 
Conversely,  if  the  problem  with  terminal  cost  can  be  solved  for  a  sufficiently  broad 
class  of  F,  one  can  derive  Theorem  2.2.  This  is  how  we  will  prove  the  theorem,  and 
moreover  the  proof  will  be  based  on  the  fact  that  finite  dimensional  representations 
analogous  to  those  in  Theorem  2.2  but  with  these  terminal  costs  are  classical  sense 
solutions  to  the  associated  PDF.  The  proof  also  has  a  number  of  side  benefits,  such 
as  convenient  representations  for  the  various  derivatives  of  the  solution  in  terms  of 
Lagrange  multipliers. 

The  calculus  of  variations  problem  (2.8)  has  a  natural  control  interpretation, 
where  9{s),  t  <  s  <Tis  the  control,  ip{s)  =  M[9]{s)  are  the  dynamics,  R  (0(s)||/9  (s,  <^(s))) 
is  the  running  cost  and  F(x)  is  the  terminal  cost.  It  is  expected  that  if  we  define 

V{x,t)=  inf  j  /  R(6'(s)||p(s,v7(s)))(is  +  F((^(r))|  ,  (2.13) 

v£C(lt,T],Sj),<p(t)=^  Ut  ) 

then  V{x,t)  is  a  weak-sense  solution  to  the  Hamilton-Jacobi-Bellman  (HJB)  equa¬ 
tion 

Wt-lH{W^,x,t)  =  0, 
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and  terminal  condition 


W{x,T)  =  F{x). 

Here  the  Hamiltonian  H{p,x,t)  is  defined  by 

H{p,x,t)  =  inf  [{p,M[e])  +  R{e\\p{t,x))] 

and  Wt  and  Wx  denote  the  partial  derivative  with  respect  to  t  and  gradient  in  x, 
respectively.  Note  that  by  the  representation  formula  [1,  Proposition  1.4.2],  the 
infimum  in  the  definition  of  H {p,  x,  t)  can  be  evaluated,  yielding 

\  W{x,T)  =  F{x) 

Note  the  use  of  the  convenient  expression  (2.5)  for  pj^{x,t). 

For  a  general  smooth  F  (2.14)  need  not  have  a  smooth  (C^)  solution.  However, 
for  affine  terminal  costs  F{x)  =  {£,x)  +  b  there  is  a  solution  (it  is  in  fact  the 
unique  solution),  and  as  remarked  above,  these  solutions  can  be  used  to  carry  out 
a  fairly  complete  analysis  of  the  problem  with  more  general  terminal  conditions. 
Indeed,  for  a  general  (proper)  convex  terminal  cost  F{x),  the  Legendre  transform 
gives  a  representation  of  the  form 

F{x)=  sup  [{f3,x)  -  h{/3)] 

for  some  proper  convex  function  h.  Let  V^{x,t)  denote  the  solution  (explicit  or 
otherwise)  to  the  calculus  of  variations  problem  (2.13)  with  terminal  cost  F{-).  Then 
one  can  show 

V^{x,t)=  sup 
/3eR^+2 

and  an  analogous  formula  for  U^{x,t)  =  mi[J'{x,t;y)  +  F{y)].  Given  Proposition 
2.4  below,  then  follows.  Since  oo  •  l{y}c  is  a  proper  convex  function,  the 

formula  can  be  extended  even  further  to  very  general  F. 

Observe  that  IF  is  a  solution  of  just  the  PDF  alone  (i.e.,  without  the  terminal 
condition)  if  and  only  if  IF  +  c  is  a  solution  for  any  real  number  c.  Since  x  is 
a  probability  vector,  it  suffices  to  prove  the  representation  under  the  conditions 
0+1  =  0  and  6  =  0. 

2.3  Explicit  solution  for  affine  terminal  costs 

Proposition  2.4.  Consider  {x,t)  £  Ra  and  F{y)  =  {£,y),  where  £  G  and 

0+1  =  0.  Define 

V{x,t)=  inf  j/  R{e{s)\\p{s,(p{s)))ds  + F{(p(T))\ , 

>p£C(lt,T],Sj)Mt)==:  Ut  J 
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and 


U{x,t)  = 


mm  X 

renx,t;T) 


Q' 


a+fc 


a  +  k 
a  +  t 


{T-t) 


(2.15) 


+  Xl+lR  (^7r^+^  Qa+r{x,t)  "  i))  )  +  >< 


where  -k  ^  !F  {x,  t;  T)  means  that  vr  satisfies  the  constraints  in  (2.11).  Then  V (x,  t)  = 
U {x,  t). 


The  proof  of  this  result  is  given  in  the  next  subsection.  We  close  this  subsection 
with  remarks  on  the  LLN  limit  distributions. 

We  will  use  the  fact  that  if  a  G  M  and  |2;|  <  1  then  the  binomial  expansion 


(1  +  ^)-“ 


is  valid,  and  if  —a  G  N  then  the  sum  contains  only  a  finite  number  of  nonzero 
terms  and  is  valid  for  all  z  G  M.  Recall  that  for  i  G  N  U  {0}  and  o  >  0,  s  >  0  or 
a  G  — N,  0  <  s  <  —a,  then 


1  + 


If  a  >  0,  s  >  0  and  \s6/ (o  +  s)|  <  1,  then  the  binomial  expansion  gives 


E  «(»)«■  =  E(- 


i=0 


i=0 


SN 

1  +  - 

a 


E 


i=0 


1  +  - 
a 


,  s  s9 

1  + - 

a  a 


1  - 


1  + 

s 

s  +  a 
s9 


-9 


s  +  a 


=  1  +  7(1-^ 


We  thus  have  the  following  expressions,  where  the  second  one  may  be  justified  by  a 
very  similar  calculation  (when  \s9/{a  +  s)|  <  1): 


CXD  OO  ^ 

J]Q“(s)0*=  (l  +  ^(l-0))““,  Y.tQ-{s)9^  =  s9{l+"-{l-9)y’'~  .  (2.16) 

2=0  2=0 

Note  also  that  when  —a  G  N  and  0  <  s  <  —a  the  number  of  nonzero  summands  is 
finite  and  the  formulas  again  hold.  If  —  o  G  N  and  0<s<— aora>0  and  s  >  0 
then  Qf{s)  >  0  for  z  G  N  U  {0}.  Letting  0  t  1  in  the  first  expression  shows  that 
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under  these  conditions  Qf  (s)  defines  a  probability  measure  on  NU  {0}.  When  a  =  0 
we  use  the  limiting  values 

Q°(s)  =  1,  Q^{s)  =  0 

for  all  i  G  N  and  s  >  0.  For  later  use  note  that  similar  calculations  show  that  if 
—a  G  N  and  0  <  s  <  —a  or  o  >  0,  s  >  0,  and  s/ {a  +  s)  <  1,  then 


^=0 


oo 

j=o 


(2.17) 


2.3.1  Analysis  of  the  finite  dimensional  minimization  problem 

We  now  focus  on  proving  Proposition  2.4.  We  will  do  so  by  proving  that  U{x,t) 
is  a  classical  sense  solution  to  the  HJB  equation  (2.14).  A  modification  of  the 
standard  verification  argument  [4]  can  then  be  used  to  show  that  V{x,t)  =  U{x,t). 
The  classical  verification  argument  consists  of  two  parts.  One  first  considers  any 
valid  occupancy  process  and  control  {(p,9)  for  the  initial  condition  {x,t).  If  t/  is  a 
smooth  solution  to  the  PDF  (2.14)  in  neighborhood  of  {((/?(s),  s)  :  t  <  s  <  T}  and  if 
U (<y?(T) ,  T)  =  {Lp{T),t),  then  the  chain  rule  implies  that  the  cost  along  this  trajectory 
is  at  least  U{x,t).  The  reverse  inequality  is  proved  by  defining  an  optimal  feedback 
control  through  the  HJB  equation,  using  this  control  to  construct  a  trajectory,  and 
then  verifying  (once  again  via  the  chain  rule)  that  the  cost  for  this  control  is  U{x,t). 
The  characterization  of  P(x,  t)  as  an  infimum  over  all  valid  occupancy  processes  and 
controls  that  start  at  {x,t)  then  gives  V{x,t)  =  U{x,t).  However,  we  have  to  clarify 
here  what  is  meant  by  a  “classical  sense”  solution  to  (2.14).  The  difficulty  is  that 
U{x,t)  is  only  well  defined  on  the  set  Va,  which  does  not  have  interior. 

Given  any  point  {x,t)  G  Va,  we  will  prove  that  one  can  extend  U{x,t)  smoothly 
to  a  neighborhood  of  {x,  t)  in  x  M.  To  be  more  precise,  for  any  such  {x,  t) 

we  will  show  there  exists  a  neighborhood  U  C  X  M  of  (x,  t)  and  a  function 

U  G  C'°°(^,M),  such  that  U{y,s)  =  U{y,s)  for  {y,s)  ^Ur\Va,  and  that  U  satisfies 
(2.14)  in  U  n  Va-  One  can  then  use  U  in  place  of  U  in  the  verification  argument, 
since  any  feasible  trajectory  will  never  leave  Va- 

To  analyze  U{x,t)  we  formulate  an  appropriate  Lagrangian.  Let 


f{x,t-,7r)  = 

7 _ r»  \ 


k=0 


Q 


a+k 


a  +  k 
a  +  t 


{T-t) 


(2.18) 


+  xi+iR  (^TT^+i  -  i))  )  +  ^  X  tt) 

and  for  a  set  of  Lagrange  multipliers  A  =  (A,  y)  =  (Aq,  Ai,  . . . ,  A/,  A/+i,  /ti),  let 


L(x,t;A;7r)  (2-19) 

7+1  /  oo  \  /  7+1  oo  \ 

=  f{x,  t;  tt)  +  E  ^kXk  I  1  -  E  ^  I  ^  ^  “  E  E'^^i  )  ’ 

fc=o  y  j=o  j  y  fc=o  i=o  j 

It  follows  from  the  definition  of  U{x,  t)  that  U{x,  t)  =  inf^r  sup^  L{x,  t;  A;  vr). 
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Note  that  by  the  joint  convexity  of  relative  entropy,  L{x,t;  A]tt)  is  convex  in 
TT.  Thus  (2.15)  is  a  standard  convex  programming  problem  with  linear  constraints, 
except  that  the  minimization  is  over  a  variable  vr  which  is  inhnite  dimensional.  Hence 
the  standard  Lagrange  mnltiplier  method  does  not  apply  directly.  If  we  temporarily 
ignore  this  issue,  then  to  guess  the  form  of  the  minimizer  one  would  of  course  set 
£)^L(x,  t;  A;  tt)  =  0  to  get  vr  =  7r{x,  t;  A),  where  D-,^  stands  for  the  gradient  in  vr  and 

7r^{x,t-A)  = 

A:  =  0, 1, . . . ,  /  and  j  >  0,  (2.20) 

7r{+i(x,t;A)  = 

\  a  +  t  J 

Here,  for  notational  simplicity,  we  extend  £  in  Proposition  2.4  by  letting  £*  =  0 
when  i  >  I.  Note  in  particular  that  {vr^}  will  depend  on  x  only  when  A;  =  /  +  1. 
Observe  also  that  setting  IlAL(a:,  t;  A;  vr)  =  0  gives  the  constraints  (2.11).  For  any 
{x,t)  G  X  M  and  A  G  M^+^,  let  7r{x,t]A)  be  determined  by  (2.20)  and  define 
G  :  M'^+2  X  X  M-f+3^  K-f+3  by 

oo  \ 

1  -  ^  (x,  t;  A)  ,  A:  =  0, 1, . . . ,  I  +  1 

i=o  / 

/+1  OO 

T  -t-Y^  xkY^'^j 

k=0  j=0 

In  the  next  theorem  we  show  that  the  '7r(x,  t;  A)  dehned  in  (2.20)  indeed  give  the 
minimizer  of  (2.15). 

Theorem  2.5.  For  any  {x,t)  G  Va  define  U{x,t)  by  (2.15).  Then  there  exists 
A  G  so  that  G{x,t;A)  =  0,  and  7r(x,t;A)  is  a  minimizer  of  (2.15).  Thus 

U{x,t)  =  L^x,  t;  A;  7r(x,  t;  A)^  .  In  addition,  the  A  that  satisfies  G(x,  t;A)  =  0  is 

unique.  Hence  if  G{x,t]  A)  =  0  for  some  A  G  then  7r(x,  t;  A)  is  a  minimizer 

of  (2.15). 

The  proof  is  divided  into  three  lemmas.  For  a  point  (x,t)  G  Va,  quantities  of 
the  following  sort  will  appear  frequently  in  the  proofs  of  the  lemmas: 

(2-21) 

.,+1  ^  /  a  +  T(x,t)  _ 

•’  \  a  +  t 

for  A:  =  0, 1, . . .  ,/;j  =  0, 1,. . . . 

In  particular,  it  will  often  be  the  case  that  (2.16)  must  be  invoked,  with  a  there 
replaced  by  o  + A:  [or  o  +  t(x,  t)]  and  s  there  replaced  by  the  corresponding  argument 
in  the  expression  above.  We  note  that  the  conditions  required  for  (2.16)  will  always 


Gk{x,t;A)  = 

Gi+2{x,t-,A)  = 
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hold  so  long  as  t  G  This  is  straightforward  to  check  in  the  case  of  a  >  0.  For 

the  case  —a  G  N,  it  nses  that  —a  =  I  +  1,  T  <  —a,  and  that  always  t{x,  t)  =  I  +  1. 
Thus  for  example  for  any  A:  G  {0, . . . ,  — o}  and  t  G  [0,T],  {a  +  k){T  —  t) / {a  +  t)  >  0 
and  (T  —  t)/{—a  —  f)  <  1  shows  that  (o  +  k){T  —  t)/{a  +  t)  <  —a  —  k,  as  reqnired 
for  (2.16). 

Lemma  2.6  (General  properties).  For  any  {x,t)  G  Va  define  U{x,t)  by  (2.15). 
Then  lF{x,t‘,T)  is  nonempty,  minimizing  measures  tt*  exist,  and  if  k  is  such  that 
Xk  >  0  and  j  G  {0, 1, . . .},  then 

Ttj>0  implies  >  0.  (2.22) 

Proof.  According  to  (2.16)  the  qnantities  in  (2.21)  are  probabilities  that  satisfy 
(2.11).  This  shows  that  J-{x,t]T)  is  nonempty.  Next  note  that  with  this  notation, 
we  can  rewrite  (2.18)  as 

f{x,  t;  tt)  =  R  {x  ^  TT  \\x  0  Tt)  +  {£,  X  X  tt) ,  (2.23) 

where  {x  (8)  7r)ij  =  Xiir).  Since  the  relative  entropy  has  compact  level  sets  in  the 
first  argnment  [1,  Lemma  1.4.3(c)],  the  existence  of  a  minimizer  of  (2.15)  follows. 
In  addition,  becanse  of  the  strict  convexity  in  that  argument  we  know  that  the 
minimizer  is  unique  up  to  those  with  Xk  >  0. 

For  a  general  initial  condition  {x,t)  let  JC  =  {k  :  Xk  >  0}.  Then  the  choice  of 
{vr.^  :  k  ^  /C}  will  not  affect  either  the  constraint  (2.11)  or  the  objective  function 
(2.23).  Hence  we  can  consider  the  equivalent  minimization  problem  over  FA(^x,t)  = 

Ivrj^  :  A:  G  /C,  j  =  0, 1, . .  .|.  As  discussed  in  the  previons  paragraph,  a  minimizer  in 
exists  and  is  nnique.  Let  this  minimizer  be  denoted  tt*  . 

Lastly  we  must  show  (2.22).  Let  =  (1  —  e)7r*  +  evf,  where  tt  is  defined  in  (2.21) 
and  let  /(e)  =  f{x,  t;  vr*^).  By  computing  the  derivative  of  /(e)  explicitly,  it  is  readily 
observed  that  if  (2.22)  does  not  hold  then  /'(e)  ^  — oo  as  e  ^  0.  Thns  (2.22)  mnst 
be  true  since  otherwise  tt*  is  not  the  minimizer.  □ 

Lemma  2.7  (Characterization  of  the  minimizer).  For  any  {x,t)  G  Va  define 
U{x,t)  by  (2.15).  Then  there  exists  A  G  so  that  G{x,t]A)  =  0,  and  7r{x,t]A) 
is  a  minimizer  of  (2.15). 

Proof.  We  want  to  argue  that  the  minimizers  must  take  the  form  of  (2.20).  However, 
there  is  a  difficulty  since  M(x,t)  can  be  infinite  dimensional.  To  deal  with  this  we 
use  a  truncation  argument  adapted  from  one  in  [2].  For  any  iV  G  N  let 

N  N 

T(^^  =  ^Xk^j7rf,  af^  =  ^TTf,  feG/C 

k&K  i=0  3=0 

and  also  let 

f^^\x,t;TT)  =  f{x,t]7r), 
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where 


TTj  =  TTj  for  k  £  }C,j  <  N 
TTj=7rf  ioT  kelCJ  >  N. 

Since  vr*  is  the  minimizer  of  (2.15)  automatically 

U{x,t)  =  min  f^^\x,t]7r),  (2.24) 

TT 

where  the  minimum  is  subject  to  the  constraints 

N  N 

X]  ^  ^  ’  /c  G  /C. 

k^K  j=0  j=0 

We  can  now  apply  the  standard  Lagrange  multiplier  method  to  (2.24).  The  first 
step  is  to  formulate  the  Lagrangian  for  this  finite  dimensional  problem: 

L^{x,  t]  A;  vr) 

(N  \  (  I+l  N 

I  +  -  jTTj 

j=0  J  \  k=0  j=0 

We  have  that  |vr*^  ■  k  £  JC,j  <  n'^  satisfies  the  constraints  in  (2.24),  and  by  (2.22) 
we  know  that  vr*^  >  0  if  vf^  >  0.  Hence  by  [6,  Corollary  28.2.2]  and  [6,  Theorem 

28.3]  applied  to  (2.24),  there  must  exist  a  set  of  Lagrange  multipliers  so 

that  the  minimizer  of  (2.24)  has  the  form 

(2.25) 

for  A:  G  /C  and  0  <  j  <  N .  li  k  +  j  >  I,  then  since  ik+j  =  0 

!Ll±i  _  c  ■ 

*k  ~  ^  ^  ’ 

vrf 

where  C  does  not  depend  on  N.  Thus  is  independent  of  N,  and  hence  is 
also  independent  of  N.  Since  the  choice  of  N  is  arbitrary,  we  then  know  that  for 
all  fc  G  /C  and  j  =  0, 1, ...,  vr*^  indeed  has  the  form  in  (2.20)  for  a  suitable  choice 
of  Afc  and  /x.  For  k  ^  )C,  we  can  simply  define  vr*^  as  in  (2.25)  and  then  solve  for 
Xk  from  the  normalization  constraint  ~  When  defined  in  this  way, 

A*  =  (Ao, . . . ,  A/+1,  /u.)  automatically  satisfies  G{x,  t]  A*)  =  0. 

For  A:  G  /C  the  corresponding  Xk  are  a  Kuhn-Tucker  vector  as  in  [6,  Corollary 
28.2.2],  and  hence  each  Xk  <  oo.  However,  for  A:  ^  /C  the  finiteness  of  Xk  is  not 
automatic. 


18 


To  show  the  finiteness,  we  first  insert  the  explicit  form  of  {x,  t;  A)  from 
(2.20)  into  Gi+i{x,t,  A)  =  0  to  obtain 

QO-+r{x,t)  ^a  +  r(x,t)  ^ 

Using  (2.16)  to  evalnate  the  sum  gives 

Az+i  =  (a  +  r(x,t))log 

For  notational  simplicity  define 

'a  +  r  —  e^(T  —  t) 


a  +  T  —  e^(T  —  t) 
a  +  t 


+  1. 


r]{t,  n)  =  log 


a  +  t 


X{x,  t;  fj,)  =  {a  +  t{x,  t))rj{t,  n)  +  1.  (2.26) 


Then  A7+1  =  A(a:,t;  fi).  Choose  C  <  00  snch  that  \£k\  <  C  for  0  <  k  <  I.  Then 


00  CXD 

j 

j=0  j=0 


j=0 


i+iM+C” 


A  calculation  of  the  same  sort  that  gave  the  display  above  (2.26)  gives 

(a  +  k)7](t,  /a)  +  1  -  C  <  Ak  <  (a  +  k)r]{t,  n)  +  1  +  C  k  =  0, . . . ,  I. 


Hence  A^  <  00  so  long  as  Aj  <  00  for  some  i  =  0, 1, 1,  which  is  true  by 
[6,  Corollary  28.2.2].  This  completes  the  proof  that  for  any  {x,t)  G  Va  there  exists 
A  G  so  that  G{x,  t;  A)  =  0  and  7r(x,  t;  A)  is  a  minimizer  of  (2.15).  □ 


The  next  lemma  will  focus  on  the  claim  that  for  {x,t)  G  "Da,  there  is  only  one  A 
that  satisfies  G{x,  t,  A)  =  0,  which  together  with  the  previons  lemma  completes  the 
proof  of  Theorem  2.5. 

Lemma  2.8  (Uniqueness  of  characterization).  For  {x,t)  G  Va,  there  is  only 
one  A  G  M.^~^^such  that  G{x,t,A)  =  0. 

Proof.  Recalling  the  definition  of  vf  in  (2.21),  notice  that  (2.20)  is  simply 

TTj{x,  t]  A)  =  for  k  =  0,1, ...  ,I  +  l,j  =  0,1, ... . 

As  noted  previonsly,  for  any  {x,t)  G  Va  we  can  assume  that  each  is  a  valid  prob¬ 
ability  vector.  Thus  iijix,  f;  A)  >  0  and  for  each  k  at  least  one  of  ^'k^{x,  t;  A),  j  =  0, 1, . 
is  strictly  positive.  If 


00  00 

ak  =  '^TTj{x,t-,A)  Tk  =  '^j7rj{x,t;A), 
j=0  j=0 
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then  ak  >  0  for  each  fc  =  0, 1, . . . ,  I,  /  +  1  and  any  A  G  Using  the  particular 

dependency  of  TTj{x,t]A)  on  A,  one  can  compute 


d'n-^{x,t;A) 

djl 

97rUx,t;A) 


TT^{x,t;A) 
jTr^{x,t;A) 
0  l^k. 


It  is  straightforward  to  construct  a  dominating  function  of  the  form  7tj-C-  for 
suitable  constants  C  and  D,  and  hence  by  the  Lebesgue  Dominated  Convergence 
Theorem  one  can  compute  D\G{x,  t]  A)  explicitly  as 


/ 

—ao 

0 

-To 

\ 

0 

— a/+i 

-Ti+i 

V 

-XqTo  ■  ■  • 

'  —xj+iTj+i 

-EitUkET=of^^i^-^ 

.t;A)  / 

Using  elementary  row  operations  to  make  the  matrix  upper  triangular,  we  see 
that  {-ak  ■.k  =  0,1,...,  1+1}  and  -  Efc=o  E^o  are 

the  eigenvalues  of  DxG{x,t-,  A).  We  have  already  observed  that  ak  >  0  for  all 
A:  =  0, +  Also,  for  every  A;  =  0, 1,. ..,/  +  !  the  Cauchy-Schwartz  inequal¬ 
ity  implies 


,i=o 


^7r^{x,t;A)  >  ^j7r^{x,t;A) 

G=o  )  \j=o 


It  is  easy  to  verify  that  the  necessary  condition  for  equality  [vr^  =  for  all  j] 
does  not  hold.  Hence  the  inequality  is  strict,  and  therefore 


OO  \ 

j=o  / 


Thus  DxG{x,  t;  A)  is  negative  definite  for  all  A  G 

Now  we  can  prove  the  uniqueness  of  A.  Suppose  there  are  two  different  Ai,  A2  G 
such  that  G{x,  f;  Aj)  =  0,  f  =  1,  2.  Define  A(e)  =  eAi  -T  (1  —  e)A2  and 


h{e)  =  {G{x,  t;  A(e)),  Ai  -  A2). 

Then  h' {e)  =  (Ai  —  A2)^  •  DaG  •  (Ai  —  A2).  Since  D\G{x,t-,  A)  is  always  negative 
definite,  h'{e)  <  0  for  all  0  <  e  <  I.  However  h{0)  =  h{l)  =  0.  This  contradiction 
shows  that  G{x,  f;  A)  =  0  has  a  unique  solution  in  A.  □ 


The  next  theorem  considers  differentiability  properties  oiU{x,t).  As  mentioned 
previously,  we  first  extend  the  definition  of  U{x,t)  to  a  neighborhood  of  {x,t)  in 
X  M,  label  this  extension  U{x,t),  and  then  show  U{x,t)  is  differentiable  in  the 
normal  Euclidean  sense.  For  our  needs  (a  verification  argument)  the  function  lJ{x,  t) 
can  be  used  in  lieu  of  U{x,t). 


20 


Theorem  2.9.  Fix  {x,t)  G  Va  and  define  U{x,t)  by  (2.15).  Then  there  is  an 
open  neighborhood  U  of  {x,  t)  and  an  extension  U  of  U  from  Va  r\U  to  U  which  is 
differentiable  on  U. 

Proof.  By  Theorem  2.5,  for  any  (x,  f)  G  Va  there  exists  A  so  that  G(x,  t;  A)  =  0  and 
U{x,t)  =  L^x,t;  A,7r{x,t;  A)^ .  A  natnral  approach  to  proving  smoothness  wonld 
be  to  apply  the  Implicit  Fnnction  Theorem.  There  is  however  a  difficulty  with 
this  approach,  owing  to  the  fact  that  the  non-smooth  function  r(x,  t)  appears  in 
the  constraints  involving  G/+i  and  G/+2.  To  avoid  this  difficulty  we  consider  an 
equivalent  but  less  obvious  formulation  of  the  constraint. 

As  discnssed  above  (2.26),  if  the  Lagrange  multiplier  A/+i  is  set  to  X{x,t;jj,), 
then  the  constraint  G7+i(x,  t;A)  =  0  will  hold  automatically.  We  will  work  with  the 
reduced  set  of  multipliers  A  =  {Aq,  . . . ,  A/,  and  the  definition 

A  (^x,  t]  A^  =  {Ao,  Ai, . . . ,  A/,  A(x,  t;  /x),  .  (2.27) 

Setting 

H (x,  t]A)  =  L  ^x,  t;  A  ^x,  f ;  A^  ;  vr  ^x,  t;  A  ^x,  f;  A^  ^  ^  (2.28) 

gives  U{x,t)  =  iL(x,f;  A). 

To  apply  the  Implicit  Function  Theorem  we  mnst  show  that  there  are  smooth 
constraints  that  characterize  A.  For  i  =  0,...,/  we  nse  Gi{x,t;A)  =  0,  where 
Gi{x,t;  A)  =  Gi{x,t;A).  These  constraints  are  equivalent  since  ttj  does  not  depend 
on  A/+1  for  k  <  I.  Since  G/+i(x,f;  A)  =  0  holds  antomatically,  we  need  only  dehne 
G/+1  so  that  G/+i(x,f;  A)  =  0  is  equivalent  to  Gj+fixfi-,  A)  =  0.  We  have 

7  OO  OO 

G/+2(x,f;A)  =  T-f-  ^Xfc^j7rj^(x,f;A)  -  x/+i  ^ i7rj+^(x, f;  A), 

fc=0  j=0  j=0 

and  thus  set 

(5/+i(x,  t]  A)  =  G/+2(x,  t]  A(x,  t]  A)).  (2.29) 

Since  vrj^(-)  does  not  depend  on  A/+i  when  k  <  I  we  abnse  notation  and  write  the 
terms  of  the  form  7r^(x,  f;  A(x,  f;  A))  as  7rj(x,t;  A).  Thus 

7  OO  OO 

G/+i(x,f;  A)  =  T  -t-  ^Xk^jTTjix,t-,A)  -  x/+i  ^  j7rj+^(x,  f;  A(x,  f;  A)). 

k=0  j=0  j=0 

Since  for  (x,  t)  G  Va  the  value  A  such  that  G(x,  t;  A)  =0  exists  and  is  nniqne,  the 
value  A  such  that  G(x,  f;  A)  =  0  exists  and  is  also  unique. 

We  have  that  D^G{x,  t]  A)  equals 

0  -To 

-ai  -Tj 

xjTj  D^G/+i(x,f;  A) 
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where  it  is  only  the  last  entry  that  must  be  identified. 

We  pause  to  introduce  a  convention  which  will  be  used  in  the  remainder  of  the  pa¬ 
per.  Whenever  a  differential  operator  of  the  form  Dx  precedes  a  composed  function, 
the  derivative  is  computed  via  the  chain  rule  for  precisely  those  arguments  where 
a  composed  dependence  on  x  is  made  explicit  in  the  notation.  Thus  in  computing 
Df^Gi+i{x,t;A)  we  use  (2.29)  and  calculations  in  the  last  section  to  get 

D^Gi+i{x,t]A) 

=  D^Gi+2ix,  t;  Ao, . . . ,  A/,  A(x,  t;  ^),  p.) 

=  D^Gi+2{x,t;Xo, . .  .,Xi,Xi+i,n)  +  Dxj^^Gi+2{x,t;  Xq,  ■  ■ . ,  A/,  Aj+i,^)  •  D^X{x,t;fi) 

7+1  oo 

=  -'^Xk'^f7rj{x,t-,A{x,t;A))  -  xj+iTi+i  ■  D^A/+i(x, t; /x). 

k=0  j=0 


It  will  be  useful  to  express  in  the  Q“(s)  notation.  We  have 


^  V  a  +  t 

Recall  that  A/+i  is  chosen  to  make  this  a  probability  measure.  By  (2.16), 

T-t 


A/+1  — 1 


=  1  + 


a  +  t 


(1  -  en 


a+r(x,i) 


Hence  using  a  little  algebra  we  can  write 


-r 


T-t 


a  +  t 


-a  —  t{x,  t) 


1  + 


a  +  t 


a+T{x,t) 


e^{T  -  t) 


a  +  T  -  ei^{T  -  t) 


—a  —  t{x,  t) 


3 


a  +  T 


a  +  T  -  et^{T  -  t) 


-a-T(x,t)-j 


^a+T(x,t)  ( e^(T-t)(a  +  r(x,t)) 


a  +  T  -  e^^{T  -t) 

Again  using  (2.16) 

^  ^  w  e^^{a  +  T{x,t)){T -t) 

T,+i  =  (x,f;A(x,f;A))  =  — — 


i=o 

Recalling  the  definition 


A(x,  t;  ^)  =  (a  +  t{x,  t))  log 


a  +  T  -  e^^{T  -t) 

a^T^j-e^  -ty 
a  +  t 


(2.30) 


a  direct  calculation  shows  D^X{x,t;  fi)  =  — T/+i,  and  hence 

7+1  oo 

Dfj,Gi+i{x,t-,A)  =  -  ^Xk^fTTjix,t-,A{x,t;A))  +  xi+iTf_^^. 

k=0  j=0 


(2.31) 
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Since  aj+i  =  1,  it  follows  that  the  eigenvalues  of  Dj^G{x,  t;  A)  are  —ao,  — ai,  •  •  •  ,  —aj 
I]a-=o  ~  Y^k=o  Yl%i  t',  A(x,  t;  A)).  By  the  same  argument  as  was 

used  for  Dj^G{x,  t]  A)  these  are  all  negative,  and  hence  D^G{x,  t;  A)  is  invertible. 

We  next  claim  that  Dji^G  is  smooth  in  {x,t)-  One  can  check  that  the  only 
potentially  difficult  component  is  D^Gi+i{x,t;  A),  and  of  this  the  only  non-trivial 
part  is 

fir^A^ix,  t;  A(x,  t;  A))  -  . 

Using  (2.17)  and  some  algebra  shows  this  term  equals 


xi+i 


E 

J=o 


-xi+i{a  +  T{x,t)) 


(t-T)e^(o  +  r) 
[e^^{t-T)  +  a  +  T]‘^' 


Although  T{x,t)  is  not  smooth  xi+i{a  +  T{x,t))  is  always  smooth,  and  thus  D^G  is 
smooth  in  {x,t)-  Note  that  the  denominator  does  not  vanish  since  >  — oo. 

Therefore  G(-;  •)  is  smooth  in  a  neighborhood  of  {x,  t;  A).  By  the  Implicit  Func¬ 
tion  Theorem,  for  any  {x,  t)  G  Va  there  exists  a  neighborhood  U  C  x  M  of  {x,  t), 
a  neighborhood  V  C  of  A,  and  a  G°°  function  g  :U  V,  so  that  A  =  g{x,  t) 
and  for  every  (y,  s)  G  U,  G{y,  s;  g{y,  s))  =  0.  Dehne 


U{y,s)  =  H{y,s;g{y,s)) 


Since  g{y,s)  is  smooth  inU,  U  G  G°°{U,M.),  and  by  Theorem  2.5  U{y,s)  =  U{y,s) 
for  (y,  s)  G  7/  n  Va-  □ 


The  next  theorem  expresses  the  derivatives  in  terms  of  the  Lagrange  multipliers. 

Theorem  2.10.  Fix  {x,t)  £  Da,  and  let  A*  be  the  associated  Lagrange  multiplier. 
We  have 

f  D,,^U(x,t)  -  D,ck+if^(x,t)  =  Al- k  =  0, 1, . . . ,  I  -  1 
\  Da;jF(x,t)  -  Da;j_^_^U{x,t)  =  -  {a  +  I)g* 

and  DtU{x,  t)  =  g*  —  y* . 

Proof.  Consider  any  point  {x,t)  G  Va  and  let  A*  be  the  associated  Lagrange 
multiplier.  By  Theorem  2.9  there  exists  U  C  x  M  a  neighborhood  of  (x,t), 

V  C  a  neighborhood  of  A*,  and  a  G°°  function  A  ■.  U  ^  V  such  that  and 

U (y,  s)  =  H{y,  s;  A(y,  s))  satisfies  U{y,  s)  =  U (y,  s)  for  any  (y,  s)  G  7/  n  Pa- 
Keeping  in  mind  the  convention  regarding  differential  operators 

Dxjj{x,t)  =  Dxf,H{x,t]A{x,t)) 

=  D^^Hix,  t;  A*)  +  Dj^H{x,  t;  A*)D^^A{x,  t). 

Thus  in  the  first  line  H{x,t;  A{x,t))  is  considered  as  the  composed  function  of  {x,  t) 
(which  by  definition  is  U{x,t)),  and  we  take  derivatives  with  respect  to  two  argu¬ 
ments  and  evaluate  at  {x,t).  In  the  second  line,  Dxf.H{x,t]  A*)  means  H{x,t;  A*) 
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is  now  a  function  of  the  independent  variables  {x,t,A*)  and  we  take  derivatives 
with  respect  to  Xk  and  then  evaluate  it  at  (rc,t;  A*).  In  all  calculations,  vectors  are 
interpreted  as  row  vectors. 

With  the  notation  established,  we  can  proceed.  Note  that  by  definition  (2.28) 
D-^H{x,t]K) 

=  {x,  t]  A;  vr)  ^x,  t;  A^  +  {x,  t;  A;  vr)  (^x,  t]  A  ^  ■ 


Since  A*  =  A(x,  t;  A*)  and  vr*  are  chosen  so  that  D-j^L  {x,t;  A*;  vr*)  =  D\L  {x,  t;  A*;  vr*) 
0,  we  have  Dj^H{x,  t;  A*)  =  0.  This  gives 

D^^U{x,t)  =  D^^H{x,t-A*),  fc  =  0,l,...,/  +  l.  (2.32) 

By  the  same  argument,  we  have  DtU{x,t)  =  DtH{x,t;  A*). 

Next,  we  insert  the  explicit  form  of  '7r(x,t;  A)  from  (2.20)  into  (2.19)  to  get 


L{x,  t]  A,  7r(x,  t]  A)) 

7+1  oo  7+1 

=  -  1+  jfi]'n-j{x,t]A)  +  AfcXfc  I  1  -  y^7r^(x,t; A) 

k=0  j=0  k=0  \  j=0 

(7+1  oo 

r  -  t  -  ^  Xfc  ^  jTTj  {x,  t;  A) 

k=0  j=0 

7  oo  oo  7+1 

=  -'^Xk'^-Kj{x,t-,A)  -  xi+i'^-Kj^^{x,t;A)  +  ^  AfcXfc  +  i^{T  -t). 

k=0  j=0  j=0  k=0 

In  the  definition  of  if(x,t;A),  A7+1  is  replaced  by  X{x,t',A)  so  that  automatically 
+  =  1-  Using  X7+iA(x,  t]  +  -  X7+1  =  x/+i(a  +  r(x,  t))r/(t,  +  from 
(2.26)  and  the  definition  of  T{x,t), 

I  /  00 

H{x,f,A)  =  Xfc Afc  +  n{T  -  ^)  -  a:fc  {x,  t;  A) 
k=0  k=0  j=0 

+  x/+i(a  +  r(x,t))?7(t,+ 

I  I  00 

=  '^XkXkA  KT  -t)  -'^Xk'^Trj{x,t;A) 
k=0  k=0  j=0 

1  \  /I 


+0  I  1  — 


yy  ]  ri{t,  +  +  \  t-'^  kxk] 


k=o  J 


k=0 


J 


ri{t,n). 


Recall  from  the  explicit  expression  (2.20)  that  for  k  =  0, 1, ...  ,1,  7rj^(x,t;  A)  does 
not  depend  on  x  or  on  A/+i.  Hence  the  x  dependence  can  be  omitted  in  7rj{x,  t;  A) 
in  the  last  display,  and  we  do  so  from  now  on. 

By  (2.32), 

Uxfc  =  Afc  -  1  -  (0  + A:)r7*,  A:  =  0, 1, . . . , /, 
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and  Uxj^^  =  0,  where  rf  =  r]{t,ix*).  This  implies  the  first  claim  of  the  theorem. 
Similarly 


DtU{x,t) 

=  DtH{x,t-A*) 

=  V*  -  IJ'*  -  Dtl'^Xk'^7rj{t;A*)  \  +  [ail  -  I 

[t^oU  j  I  V  ^  /  V  ^  yj 

=  1]*  -  n*  -  Dtl  ^Xk'^7Tj{t-,A*)  i  +xi+i{a  +  T{x,t))Dtr]{t,iJ*). 

f  fc=0  j=0  J 

We  now  will  verify  that 

-Dt  I  ^Xk'^TTj{t;A*)  i  +  xi+i{a  +  T{x,t))Dtri{t-,  iJ.*)  =  0,  (2.33) 

[  k=0  j=0  ) 

which  implies  DfU (x,  t)  =  rj*  —  n*.  By  the  explicit  formnla  for  (t;  A)  in  (2.20)  and 
the  definition  of  Qf(s),  for  all  A:  =  0, 1, . . . ,  I;  j  =  0, 1, . . . 


-LytTTj  (t,  A)  JTTj  (t,  A)  rTi\/  I  J.\  A  „  I  j.  ^7  (A)  A). 


(t  —  T)(a  +  t)  a  +  t  ^ 

A  suitable  dominating  fnnction  can  be  found,  and  thus  by  the  Lebesgue  Dominated 
Convergence  Theorem 

I  OO 


Dt  \  ^Xk^7rj{t-,A) 
k=0  j=0 

I  OO  r 


fc=o  i=o 


j7r^(t;A)- 


CL  T 


+ 


{a  +  k  +  j)  k 


<(A;A) 


^  ’  ^(t  — T)(a  +  t)  a  +  t  ^ 

Using  that  7r(x,  t;  A*)  satisfies  the  constraint  (2.11)  and  some  elementary  algebra, 


{I  OO 

A* 

fc=0  j=0 

_  Xi+i{a  +  T(x,t)) 

t  —  T  a  +  t 

Equation  (2.30)  and  the  definition  of  rj  in  (2.26)  give 

Tf+i  =  '^^^{a  +  T{x,t))eD-^\ 
a  +  t 


and  hence 


I  OO 

Dt  <j  ^rcfc^7rj=(t;  A* 

k=0  j=0 


a  +  t{x,  t) 
a  +  t 


xi+i  ( 1  -  e^‘ 


(2.34) 


(2.35) 
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The  definition  of  r]{t,  jj,)  in  (2.26)  and  (a  +  t)e^  =  o  +  T  —  e^(T  —  t)  gives 

gM  I  gM-f?  _  I 

Dtr]{t,  n)  =  — — — — — — - —  = - 

a  +  i  —  (i  —  a  +  t  a  +  t 

Finally,  combining  this  with  (2.35)  gives  (2.33).  □ 


Our  final  theorem  shows  that  U  satisfies  the  HJB  equation  (2.14)  in  the  classical 
sense  on  U.  When  combined  with  a  standard  verification  argument  as  in  [4],  this 
will  imply  V{x,t)  =  U{x,t)  =  U{x,t)  on  Va- 

Theorem  2.11.  U  satisfies  (2.14)  onVa- 

Proof.  Having  derived  various  expressions  for  the  derivatives  of  U  in  terms  of  the 
Lagrange  multipliers  in  Theorem  2.10,  to  show  that  U{x,t)  satisfies  the  PDF  (2.14) 
it  remains  to  show 


a  +  t  n  -i-+  n  -i-  + 


k=0 


a  +  t 


a  +  t 


Recall  from  (2.20)  that  for  A:  =  0, 1, . . . ,  /  and  j  >  0 


TT 


*k 


=  Q 


a-\-k 


a  +  k 
a  +  t 


(T-t) 


A*-l+iM*-4+i 


e  k 


Using  the  definition  of  for  /c  =  0, 1, 1 


+k^l  (j  +  m 


I  j.\ 


Now  sum  both  sides  from  j  =  0  to  oo  and  use  the  fact  that  J^'jLo  =  1  to  get 

ia  +  t)ET=i)^t 


gAfe-A 


fc+1  =  g  ^ 


(T  -t){a  +  k) 


for  A:  =  0, 1, . . . ,  / 


(2.37) 


Inserting  (2.37)  into  (2.36),  a  little  algebra  shows  that  satisfaction  of  the  PDF  is 
equivalent  to 

I-l  oo  j- 

[T-t)  =  y^Xk+j+  +  xi- - 

LS  fii  “  +  * 

+x,+i^±^^(T  -  t)e-^‘+^‘.  (2.38) 

a  +  t 

Since  vrf  =  Q“+^  [^{T  -  A))  for  j  >  1,  by  (2.16) 

oo  oo  /  I  T  \ 

T,+‘  = 

i=i  j=i  ^  A 

=  — — +  (2.39) 

a  +  t 
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where  the  last  equality  uses  the  definition  of  rj  in  (2.26). 

Inserting  (2.34)  and  (2.39)  into  X]fe=o  =  T  —  t,  we  then  have 


i-i 


(T-t)  =  +  x/^(T-t)e^^i-’'*(“+"+i)+^ 


k=0  j=0 


a  +  t 


o  +  r^ 
a  +  t 


We  have  thus  verified  (2.38),  which  completes  the  proof  that  U  satisfies  (2.14).  □ 


2.4  Minimizing  Trajectories 

The  minimizing  trajectories  associated  with  the  calculus  of  variations  problem  have 
important  qualitative  and  computational  uses.  Perhaps  the  most  important  is  that 
they  identify  the  most  likely  way  a  rare  event  will  occur  [5] . 

Identification  of  the  minimizing  trajectories  in  the  MB  case  was  done  in  [2]  using 
the  Euler-Lagrange  equations,  a  system  of  nonlinear  ordinary  differential  equations. 
The  solutions  to  these  equation  are  called  extremals,  and  in  general  being  an  ex¬ 
tremal  is  neither  a  necessary  or  sufficient  condition  for  minimality.  In  [2],  a  direct 
but  detailed  argument  using  Lagrange  multiplier  techniques  was  used  to  show  the 
extremals  were  indeed  minimizers.  Here  we  take  a  different  tack.  We  start  in  [2] 
with  a  two-parameter  family  of  solutions  to  the  Euler-Lagrange  equations  that  iden¬ 
tify  the  extremals.  The  two  parameters  are  themselves  characterized  as  the  solution 
to  a  pair  of  nonlinear  algebraic  equations.  These  parameters  and  the  form  of  the 
extremals  suggest  values  for  the  Lagrange  multipliers  in  the  explicit  representation 
(2.15),  and  indeed  it  is  shown  that  characterizing  equations  G{x,t]A)  =  0  for  the 
unique  Lagrange  multipliers  are  satisfied.  Having  identified  the  minimal  cost,  all 
that  remains  is  the  show  that  the  cost  along  the  extremal  is  the  same  as  this  mini¬ 
mal  value.  This  is  done  by  explicitly  evaluating  an  integral. 

The  main  goal  of  this  section  is  to  argue  that  the  extremals  are  minimizers  and 
exhibit  the  relation  between  the  two  parameters  used  to  identify  the  extremals  and 
the  Lagrange  multipliers  used  in  the  formula  for  the  minimal  cost.  Not  all  details  will 
be  given,  and  to  simplify  the  presentation  only  the  initial  condition  with  t  =  0  and 
all  cells  empty  is  considered.  The  statement  of  the  case  of  general  initial  conditions 
is  exactly  analogous  to  Theorem  2.8  in  [2],  and  any  details  that  are  omitted  are 
similar  to  ones  appearing  in  [2].  In  addition,  we  present  only  the  case  a  G  (0,  oo).  In 
this  case  it  is  simpler  to  work  with  an  infinite  dimensional  version  of  the  extremals. 
This  is  analogous  to  what  is  called  exponential  case  in  [2].  The  arguments  for  the 
cases  0  =  00  and  o  <  0  are  analogous  to  that  of  o  G  (0,oo).  As  noted  above,  the 
case  0  =  00  has  already  been  considered  in  [2].  In  the  case  o  <  0,  the  extremals 
satisfy  ipi  =  0.  Because  of  this  the  arguments  are  somewhat  simpler  than  in  the 
case  of  o  >  0,  and  it  is  analogous  to  the  polynomial  case  of  [2]. 

In  the  case  of  the  empty  initial  condition  the  extremal  can  be  identified  as  follows. 
Let  y  G  Si  he  given.  Then  a  family  of  solutions  to  the  Euler-Lagrange  equations  for 
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the  problem  of  minimizing  the  cost  subject  to  this  terminal  condition  are 


Mt)  =  CQ^,{pt)  +  Y,iyk-CQt{pT)) 

k=0 

I 

Pi+{t)  =  l-^ipi{t). 

i=0 

This  is  exactly  analogous  to  the  form  found  in  [2]  for  the  special  case  of  a  =  oo, 
with  the  Poisson  distribution  V  {t)  in  that  case  replaced  by  the  family  of  probability 
distributions  It  is  useful  to  extend  the  definition  of  to  (pi{t)  =  CQf{pt) 

for  i  >  I,  while  maintaining  the  distinction  between  pi+{t)  and  (pi{t). 

The  parameters  p  >  0  and  C  >0  are  chosen  so  that  the  measure  corresponding 
to  ^i{T)  is  a  probability  measure,  and  moreover  one  for  which  the  number  of  balls 
per  cell  at  time  T  equals  T.  Specifically,  p  is  chosen  so  that 

pT-T/i=o^QlipT)  T-Y!i=,tyi 
^-T.LompT)  i-E-=oy* 

holds,  and  then 

^  1  -  E[=o  yi  ^  T  -  iVi 

1  -  E[=o  QKpT)  pT  -  E[=o  ^mpT)  ■ 

Solutions  to  these  equations  exist  for  p  G  (0,  oo)  and  C  G  [0,oo),  and  are  unique. 

To  show  that  this  is  indeed  a  minimizing  trajectory  we  relate  the  constants  p 
and  C  to  the  Lagrange  multipliers  appearing  in  the  finite  dimensional  representation 
(2.15).  Recall  that  the  minimizer  to  this  problem  takes  the  form 

7r°(l,0;A)  = 

with  (.j  =  0  if  j  >  I.  Using  the  form  of  the  minimizing  trajectory,  at  time  t  =  T 

Pj{T)  =  2/j,0<j<I, 

Pj{T)  =  CQ]{pT),I<j. 

Thus  for  all  j  >  I  we  will  want 

ga(j.)gAo-l+iM  ^  CQ'^ipT). 

Since 

QjjpT)  ^  (^)  ^  •  /  o  +  T  y  /  g  +  T  y 

^Ka  +  pTJ  \a  +  pT) 
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this  suggests 


Aq  —  1 


log  p  +  log 


Qi  -\-  T 


a  +  pT 


a  log 


a  +  T 
a  +  pT 


+  logC7 


and 

4  =  -  log  Vk  +  log  QUpT)  +  log  C 

when  k  <  I.  Hence  the  minimizing  trajectory  for  a  problem  with  a  finite  terminal 
cost  i  will  terminate  at  a  point  y  with  each  >0,  an  assumption  we  make  for  the 
rest  of  this  section.  The  argument  when  one  or  more  yk  =  0  can  be  handled  by  a 
limiting  argument.  We  remark  in  passing  that  similar  considerations  allow  one  to 
explicitly  identify  the  Lagrange  multipliers  for  all  initial  conditions  {x,  t)  that  lie  on 
the  extremal  in  terms  of  C,  p,  and  the  values  yk- 

To  show  that  7r°(l,0;A)  is  the  minimizing  probability  measure  in  (2.15)  the 
constraints  (2.11)  must  be  demonstrated.  One  constraint  is  that  7r°(l,0;A)  be 
a  probability  measure.  Since  the  definitions  of  the  Lagrange  multipliers  enforce 
7r^(l,0; A)  =  ipj{T),  this  follows  from  Y,i=oyi  =  1-4  + 


OO 


I  OO 


=  E  CQ-{pT)  =  l-C  +  C  =  l. 

j=0  j=o  j=i+i 


The  only  other  constraint  to  check  is  the  conservation  condition: 


OO 


j=0 


j=0 

OO  II 

E  j4Q“(pr)  +  E 

j=0  j=0  j=0 

I  I 

CpT-^jCQ-{pT)  +  ^jy, 
1=0  1=0 


I  I 

T  -'^iyi  +  '^jyj 

i=0  1=0 

T, 


where  the  equations  characterizing  C  and  p  are  used  for  the  fourth  equality. 

We  have  identified  the  optimal  measure  for  the  terminal  cost  To  complete  the 
argument  that  99  is  a  minimizer  we  need  only  show  that  the  cost  along  this  trajectory 
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equals 


i?(7rO(l,0;A)||Q“(r)) 


oo  /, 

^7r°(l,0;  A)log  - 

1=0  V 


j=0 

oo 


7r°(l,0;  A)' 
Q%T) 


i=o 

/ 


%•  log 


Vi 


j=0 


Q%T) 


+  E  CQ']{pT)log 

j=i+i 


CQ^,{pT) 

Q%T) 


j=0 


Q“(r) ; 


V^,(r)  log  ((^,(T))  -  (^.(T)  log  (Q“(r)) 

j=0  j=0 

oo 

J]v^,(r)iog((^,(T)) 

i=o 


Y.Vi(T) 


j=0 


rV 

log  (  -  1  +  log 


nLo(«+^) 


+  log 


a  +  T 


-a-:/ 


E  (V^j(2'))  -  T'log  +  (a  +  T)  log 


o  +  T 


'^Pj{T)  log 
j=o 


Uillia  +  k) 


The  notion  of  a  completely  monotone  function  is  useful  here.  Although  it  is 
clear  from  the  construction  of  the  Lagrange  multipliers  that  Pj{T)  is  a  probability 
measure,  the  same  cannot  be  said  yet  for  (pj{t).  A  monotone  function  7  is  completely 
monotone  on  [0,  T]  if  it  is  infinitely  differentiable  on  [0,  T]  and  if  for  all  t  G  [0,  T] 
and  i>  0 

(-l)V*Hi)  >  0. 

The  same  argument  as  [2,  p.  2794]  shows  that  poi^)  is  completely  monotone  on 
[0,  T],  and  hence  for  all  t  G  [0,  T]  and  i  >  0,  Pi{t)  >  0.  From  the  explicit  formula  for 
Po{t)  we  actually  have  (— l)Vo  ^  {t)  >  0  for  t  G  (0,  T).  It  is  also  easy  to  show  that  for 
all  t  G  [0,  r]  and  i  >  0,  pi{t)  can  be  interpreted  as  the  ith  term  in  the  Taylor  series 
expansion  of  (^o(O)  about  t,  and  so  for  each  t  {pi{t),i  =  0, 1, . . .}  is  a  probability 
measure  on  {0, 1, . . .}. 

To  evaluate  the  cost  ^ 

/  R{0\\p{t,Pit)))dt, 

Jo 


30 


it  is  convenient  to  work,  as  in  [2],  with  the  cumulative  occupancy  functions 


i=0 


The  dynamics  then  take  the  form  {t)  =  and  so  with  the  convention 

'Ip-lit)  =  0  the  cost  can  be  expressed 


JO 


-pf\t) 


i=0 


{pi{t)  -  pi-i{t)) 


+  {^  + 'Po\t)  +  ■  ■  ■  + 

\ 

We  have 


1  +  p^^\t)  +  ■  ■  ■  + 


^  (1  -  pi{t))  +  ^  E^7+i  k  iPkit)  -  Pk-lit)) 


dt. 


fc=0 


and  so 


Since  for  i  >  / 


‘Piit) 


it) 


PP’it) 


Po\t) 


^^ipiit)  -  Pi-lit)) 


and 


'^  +  Po\t)  +  ■  ■  ■  +  PY’ it) 


a  +  t  —ipQ^^\t) 

"  +  *  ^‘o\t) 

a  +  t  Q'i+iipt)ii  +  1) 

a  +  i  Qfipt)t 
a  +  t  i—pt)i—a  —  i) 
a  +  i  (a  +  pt)t 
(a  +  t)p 
a  +  pt 

(1), 


^  (1  -  piit))  +  ^  Y)kLi+i  k  iPkit)  -  Pk-lit)) 

+,  (E”,+i  +  +  E£/+.  J-++\t} 

ES,+i 


a+i  /  ^;poo  j-ty  ,Pi) 
a-\-t  \2^i=I+l  i\  rO 
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we  can  write  the  integral  as 


i=0 


lo 


a-\-i 


a+t  (V’i(i)  -  V’i-l(i)) 


dt. 


At  several  places  below  we  will  need  the  existence  of  a  dominating  function  to  justify 
the  interchange  of  summation  and  integration.  A  suitable  function  can  be  found 
using  the  same  calculations  as  those  used  to  establish  (2.16).  Also,  this  dominating 
function  will  work  only  on  closed  subintervals  of  (0,  T),  and  so  a  careful  argument 
will  first  evaluate  the  integral  on  [e,  T  —  e]  and  then  use  monotone  convergence  to 
take  the  limit  e  |  0. 

We  evaluate  the  integral  by  the  following  calculation,  each  line  of  which  is  ex¬ 
plained  below: 


pT  oo 


log 


a  +  t  (^) 


dt 


i=0 


®  {i) 

E  (0  +  V'f  ^  (^)  log  i’o^  (t)  -  (i)  log  (a  +  f ))  dt 


rT 


i=0 

fT  /  ^ 


+  /  (  E  (a  +  t)  dt 


Jo 

rT 


,i=0 


y 


E  “  y’S V'? (i)  -  (^)  log  (a  +  i) 


•^0  i=0 

+  /  log  (a  +  t)  dt 
Jo 


dt 


OO  /  'J~'  /  oo  \  oo 

E  f  (t)  Q  +  /  (  ^  j  ^  log(a  +  i) 


+(a  +  T)  log(a  +  T)  —  (o  +  T)  —  a  log  o  +  o 


E  (  ‘f^»(^)log  +o\'^)  -Ai(0)log  +o\o)  ^  +log(a  +  f) 


i=0 


Kk=0 


k=0 


+T  +  {a  +  T)  log(a  +  T)  -  (a  +  T)  -  a  log  a  +  a 

E  (  +iiT)  log  {MT)i\/r)  -  log(a  +  i)  (  E  )  ) 

^=0  \  \k=i-\-l  /  / 


+{a  +  T)  log(a  +  T)  -  o  log  a 


'i-i 


E  +iiT)^ogifi{T)  +  ^(pi{T)  log  {i\)  -  (pi{T)  log  (  JJ(o  +  fc) 

i=0  V  i=0  \k=0  / 

—T  log  T  +  {a +  T)  log(a  +  T)  —  a  log  a 

OO  OO  /  ..  \ 

'^ipi{T)  log  ipi{T)  +  '^ipi{T)  log 


nfc=o(«  +  ^). 


i=0 


i=0 


—T  log  T  +  {a +  T)  log(a  +  T)  —  a  log  a. 
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The  first  line  separates  the  a  +  t  term  in  the  logarithm.  The  second  equality  uses 
the  convention  =  0  and  that  ~  ^  balls  go  into  some 

cell).  The  third  line  nses  integration  by  parts  and  the  fonrth  nses  the  definition 
of  the  cumulative  occupancies.  The  fifth  line  uses  the  definition  of  ipi  in  terms  of 
derivatives  of  tpo,  and  the  sixth  equality  uses  summation  by  parts.  Since 


-riog 


+  (a  +  T)  log 


-T  log  T  +  (a  +  T)  log(a  +  T)  -  a  log  o, 


the  cost  along  this  trajectory  eqnals  the  minimum,  and  the  argument  is  complete. 

□ 


3  Allocation  Models  with  Balls  of  Different  Color 

In  this  section  we  extend  the  techniqnes  to  the  case  of  allocation  models  where  the 
balls  are  of  more  than  one  type.  To  keep  the  notation  simple,  we  actually  consider 
just  two  colors,  the  extension  to  the  more  general  case  being  straightforward.  An¬ 
other  simplification  is  that  we  consider  only  MB  statistics.  The  interested  reader  can 
combine  the  methods  from  this  section  and  the  last  to  treat  more  general  statistical 
models. 

3.1  Coloration  Process 

We  construct  an  allocation  model  with  colored  balls  as  follows.  Balls  are  thrown 
into  one  of  n  cells  sequentially.  The  throwing  process  is  modeled  by  a  collection 
of  independent  and  identically  distribnted  (iid)  random  variables,  each  nniformly 
distributed  on  the  set  {1, . . .  ,n},  with  each  value  of  the  set  corresponding  to  an 
cell.  There  is  also  a  coloration  process.  At  each  discrete  time  a  ball  is  assigned 
color  Ij”  G  {1,  2},  and  then  placed  into  the  cell  determined  by  the  throwing  process. 
These  two  processes  are  independent. 

The  occnpancy  process  in  this  case  is  defined  as  follows.  The  natnral  state  space 
is 

{/+!  J+i 

X  G  X  :  xij  >0,0<f<I  +  l,0< 

i=0  j=0 

If  i  G  {0, . . . ,  1}  and  j  G  {0, . . . ,  J},  then  Xf-{l/n)  is  the  fraction  of  cells  containing 
exactly  i  color-I  and  j  color-2  balls  when  I  balls  have  been  thrown.  In  an  analogous 
fashion  Xf_^  -{I / n) ,  Xf  jj^{l / n)  and  jj^{l/n)  are  defined.  By  definition  Aq  q(0)  = 
1,  and  AT(0)  =  0  for  all  other  valnes  of 

To  describe  the  large  deviation  asymptotics  of  these  allocation  processes  we 
mnst  specify  those  of  the  coloration  processes.  Cumnlative  coloration  processes 
{r"',  n  G  N}  are  defined  for  t  =  l/nhy 

ri{l/n)  =  -  X]  ^2  (V”)  =  “  X]  ^{Tr=2}- 

k=l  k=l 
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We  will  assume  that  these  processes  satisfy  a  large  deviations  principle  with  a  rate 
function  of  the  form  J{4>)  =  fg  c((^(s))ds.  Thus  c((p{s))  is  a  measure  of  the  local  (in 
time)  log  likelihood  that  a  fraction  4>i{s)  of  the  balls  are  color  i.  A  mild  technical  as¬ 
sumption  that  is  needed  to  prove  a  large  deviations  result  for  the  occupancy  process 
is  that  c(a)  =  0  for  some  point  a  with  a*  >  0,  i  =  1,2.  Since  c  is  a  rate  function, 
there  is  at  least  one  probability  vector  a  at  which  c(a)  =  0.  The  assumption  that 
this  occurs  at  a  point  where  both  components  are  positive  is  very  mild,  and  means 
simply  that  the  LLN  limit  cannot  concentrate  exclusively  on  one  color. 

Examples  of  coloration  processes  which  satisfy  these  properties  are  deterministic, 
iid  and  Markovian.  In  the  iid  case  colors  are  selected  by  an  iid  sequence  of  random 
variables.  In  the  Markov  case  the  color  is  chosen  by  a  finite  state  ergodic  Markov 
chain.  The  so-called  deterministic  case  seeks  to  achieve  a  deterministic  fraction 
of  color  k,  with  Uk  G  (0,1).  This  can  be  done  as  follows.  If  Nj^_i  balls  of  color 
k  have  been  thrown  in  the  first  I  —  1  throws  (with  =  ^  —  1),  and  if 

<  ail/n,  then  we  color  the  /th  ball  1,  and  otherwise  color  it  2. 

The  specific  form  for  c  in  all  these  cases  is  spelled  out  in  [3].  For  reasons  to  be 
explained  below,  the  focus  in  this  paper  will  be  on  the  iid  and  deterministic  cases, 
where  c{p)  equals  R{p  ||a)  and  oo  •  l^aYip)-,  repsectively.  Under  a  snitable  restriction 
needed  to  ensnre  convexity  that  is  also  described  below,  the  same  methods  can  be 
applied  to  the  Markovian  case  as  well. 

3.2  Variational  Problem  and  PDE 

For  this  problem  the  feasible  domain  is 

r  I  j 

V  =  ■j  (x,  f)  G  Si^j  X  [0,  T)  :  ^  ^  Xi+ij  +  xj+yj+i  >  0  and 

[  i=0  i=0 

/+1 J+1  I  J  I  J 

t  >  EE  Xjj  or  -I-  -t-  x/+i,j+i  =  0  and  t  =  EE  xpj 

i=0  j=0  i=0  j=0  i=0  j=0 

We  next  describe  the  large  deviations  variational  problem.  Recall  that  Si^j  are 
the  set  of  all  probability  measures  on  {0,/  -1-1}  x  {0,  J  -f  1},  which  can  also  be 
interpreted  as  real  (/  -|-  2)  x  ( J  -|-  2)  matrices.  For  a  G  Sjj  define  the  linear  maps 

The  rate  function  for  the  coloration  over  an  interval  [t,  T]  is  assnmed  to  be  of  the 
form  j)  c{p)ds,  where  p{s)  =  {pi{s) ,  p2{s))  are  the  colored  fractions  at  time  s.  The 
local  (in  time)  and  total  coloration  fractions  satisfy  qk  =  pk{s)ds/[T  —  t],  and 
for  a  trajectory  of  the  form  (/3i(s),  /92('S))  =  (gi,  ^2),  the  cost  is  of  course  [T  —  f]c((7). 
The  rate  function  for  the  occupancy  process  on  path  space  is  then 

J(x,  f;  v?)  =  inf  /  [piR{9^  \\(p)  +  p2R{9‘^  \\(p)  +  c{p)]  ds, 

.Jt 
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where  the  infimum  is  over  all  0,  p  such  that 

pu 

(p{u)  -  ip{t)  =  J  [piM^[9^]  +  p2M‘^[6‘^])  ds. 
For  a  terminal  cost  F  we  consider 


V (x,  t)  =  inf 

ip&C{[t,T],Si^j),ip(t)=x 

Then  V  should  be  a  weak-sense  solution  to 


{I(x,t;p)  +  F(^(T))}. 


Wt  +  ff(W^,x,t)  =  0 
and  the  terminal  condition,  where 

H{p,x,t)  =  inf  [(p,  piM^[9^]  +  p2M‘^[9‘^])  +  piR{9^  ||a:)  +  P2R{0‘^  ||a;)  +  c{p)]  . 
pF 

If  6(7)  is  the  Legendre  transform  6(7)  =  sup^  [(7,p)  —  c{p)],  then  we  can  also  write 

-  E  fc(mf[(p,M”|0+fl(«”l|.nl)-c(rt 

m=l,2  ^  ^ 

=  —h  [  —  inf  [(p,  M^[9^])  +  R{9^  ||x)]  ,  —  inf  [(p,  +  R{9‘^  ||a;)]  )  • 

The  variational  formula  for  exponential  integrals  in  terms  of  relative  entropy  [1, 
Proposition  1.4.2]  asserts  that 


H{p,x,t)  =  —sup 

p 


inf  [(p,  [0^]  )  +  R{9^  ||rc)]  =  inf 


=  inf 
6(1 


^  Pi,j0lj  +  Ri0^ 


Y  (Pi+^d  -  11^) 


log  J] 


O  {Pi  +  l,j  Pi,j)q,.  . 


dd,i<I 


+  E  ^ 

i,j, 1=1+1 


Using  the  analogous  formula  for  m  =  2,  one  obtains 


H{p,x,t)  =  —b 


log  I  Y^  ®  +  Y^  ^ 

log  I  Y^  ^  ; 


(3.1) 
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3.3  Explicit  Solution 

Let  TTij{ri,r2)  denote  the  probability  of  throwing  additional  balls  of  color  m, 
m  =  1,  2,  into  cells  of  category  (i,  j),  and  let  q  =  {qi,  q2)  be  the  fraction  of  balls  of 
colors  (1,2).  For  x  G  5/^j,  we  say  that  (vr,  g)  G  J^{x,t',y,T)  if  for  all  i,j 


X 


OO  /+1  J  +  1  CXD 

Y.  ^idiri,r2)  =  xij,  Y  j  (ri ,  r2)  —  qm(T  t) 

i=0  j=0  ri,r2=0 


ri,r2=0 

for  m  =  1,  2  and 


k  I 

yk,i  =  -  j) 

2=0  j  =  0 

OO  /+1  I 

yi+i,i  =  EEE-«  TTijil  +  l-i  +  r,l-j) 

r=0  2=0  j=0 
OO  k  J+1 

yk,j+i  =  EEE-..  TTij{k  -i,J  +  l-  j  +  r) 

r=0  2=0  j=0 
OO  OO  7+1  J+1 

?//+!, J+1  EEEE-«  TTij  [I —  +1  —  j  +  r). 

s=0  r=0  i=0  j=0 


We  also  denote  y  by  x  x  vr.  If  the  coloration  turns  out  to  be  (51,^2),  then  there 
are  qm{T  —  t)n  balls  of  color  m  thrown,  and  the  law  of  large  numbers  limit  for  the 
empirical  fraction  of  cells  of  category  {i,j)  is  Vi{qi{T  —  t))Vj{q2{T  —  t)). 

The  same  sort  of  argument  as  in  Section  2.2  then  suggests  that  the  explicit  form 
for  the  solution  to  the  variational  problem  should  be 


1 


7+1  J +1 


mm 

in,q)£T{: 


r™  I  +(91+-^))  X  'P{g2{T  -  t)))  +  {T-t)c{q) 


However,  an  interesting  feature  of  the  case  with  color  is  that  the  quantity  being 
minimized  in  this  formula  is  not  always  convex.  In  a  previous  paper  [3],  a  useful 
assumption  that  guaranteed  the  convexity  of  the  large  deviation  rate  on  path  space 
was  that  c{p)+h{p)  be  convex,  where  h{p)  is  the  entropy  function  h{p)  =  —pi  log  pi  — 
P2^ogp2-  We  will  show  that  this  same  condition,  not  surprisingly,  gives  convexity 
here  as  well.  Let  a  be  the  point  with  Uj  >  0,z  =  1,2,  for  which  c(a)  =  0.  Then  we 
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can  write,  under  the  constraint  that  relates  and  qm, 


1) 


I+l  J+1 

EE  XijR  {TTij  \\V{qi{T  -  t))  X  Viq2iT  -  t)))  +  {T  -  t)c(g) 

2=0  j=0 

7+1  J+1  OO  CxD 

=  TTi,j{k,l)log 

i=0  j=0  k=0  1=0 

7  +  1  J+1  CxO  OO 

7r,+fe,0 

i=0  j=0  k=0  1=0 

7+1  J +1 


iPfc(ai(r-t))PK«2(T-t)) 


(T  -  t)c{q)  -  log 


Si 

ai 


ik 


02 


EE  XijR  |  +  (oi(r  -  t))  X  P(o2(T  -t)))  +  {T-  t)  [c{q)  -  R{q  ||o)] 

i=0  j=0 


The  mapping  q  — >  [0(9)  —  R{q  ||a)]  is  convex  if  and  only  if  c{q)  +  h{q)  is  convex,  and 
so  convexity  of  c{q)  +  h{q)  is  sufficient  for  the  minimization  problem  to  be  convex 
in  (Tfijjqm)-  Note  that  in  the  deterministic  case  this  condition  holds  with  strict 
convexity,  and  that  in  the  iid  case  c{q)  +  h{q)  is  convex  but  never  strictly  convex 
(it  is  in  fact  always  linear  in  q).  Hence  this  is  in  a  certain  sense  a  borderline  case, 
and  one  for  which  there  may  be  nonuniqueness  of  minimizers.  In  the  case  of  Markov 
coloring  the  condition  may  or  may  not  hold-see  [3]  for  further  details. 

This  alternative  rewriting  of  the  objective  function  also  has  a  practical  benefit, 
in  that  the  quantity  to  be  minimized  is  now  the  sum  of  a  convex  function  of  vr  and 
a  convex  function  of  q,  with  no  “cross  terms.”  As  a  consequence,  the  formulas  for 
TTij  and  qm  also  separate,  and  hence  can  be  solved  for  explicitly  in  terms  of  the 
multipliers. 

Having  restricted  already  to  the  case  where  c{q)  +  h{q)  is  convex,  we  now  make  a 
final  restriction.  To  parallel  the  very  explicit  computations  of  the  single  color  model, 
we  need  a  specific  form  for  c,  and  in  particular  a  form  that  allows  us  to  solve  for  the 
minimizers  in  terms  of  multipliers.  This  can  be  done  when  the  rate  function  for  the 
coloration  has  a  representation  in  terms  of  relative  entropy,  which  is  the  case  for  all 
the  models  introduced  previously.  The  particular  form  we  choose  is  c{q)  =  bR{q  ||a), 
where  6  G  (1,  00).  The  limit  6  t  00  gives  the  deterministic  coloration  with  parameters 
oi  and  02,  and  the  limit  b  [1  gives  the  iid  coloration  with  parameters  oi  and  02. 

Define 

J{x,t-,y)=  inf 

f{t)=x,f{T)=y 

Theorem  3.1.  Consider  the  allocation  problem  with  either  the  deterministic  or 
iid  coloration  process  with  parameters  oi  >  0  and  02  >  0,  an  initial  condition 
{x,t)  G  V,  and  a  feasible  terminal  condition  y.  Then  the  quantity  J  {x,t;y)  has  the 
representation 


mm 

{n,q)£Tt 


I 


7+1  J +1 


j  \\Riai{T-t))  X  V{a2{T  -  t)))  +  {T  -  t){b  -  l)R{q\\a) 
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Proof.  We  will  prove  the  representation  for  b  G  (l,oo).  Taking  limits  and  using 
monotonicity  in  b  will  then  establish  the  corresponding  result  for  6=1  and  6  =  oo. 
The  same  line  of  argument  as  in  the  single  color  case  is  followed.  Hence  we  consider 
linear  terminal  conditions  F{y)  =  {y,  i)  with  £  =  £i,j,i  =  0, . . . ,  /+1,  j  =  0, . . . ,  J+1, 
and  dehne 

r  i+i  J+1 

U{x,t)  =  inf  i7rij\\V{ai{T-t))xV{a2{T-t))) 

I  i=0  j=0 

+  (T  —  t)(6  —  l)R{q  ||a)  +  (f,  x  X  tt)  I . 

The  infimum  is  over  which  is  defined  to  be  the  set  of  all  collections 

{■^i,j,qm)  such  that 

7+1  J+1  oo  oo 

rmVTij (r  1,1-2)  =  qmiT  -  t),m  =  1,2 

i=0  j=0  ri=0r2=0 


and 


qm>0,m  =  l,  2,  qi+q2  =  1. 


To  study  this  problem  define 


f{x,t;-K,q) 

7+1  J +1 

=  EE  Xi^jR  (vTi J  ||P(ai(r  -  t))  X  V{a2{T  -t)))  +  {T-  t){b  -  l)R{q  ||a)  +  +  x  x  +  , 

i=0  j=0 

introduce  Lagrange  multipliers  A  =  =  0, . . . ,  I  +  1,  j  =  0, . . . ,  J  +  1;  Hm,  tti  = 

1,  2;  0),  and  define 


7+1  J +1 


L{x,t-,A,7r,q)  =  /(x,  f;  tt,  g)  +  ^  ^  XijXij  1  “  X]  X]  0 


2=0  j=0 


fc=0 /=0 


(7+1  J+1  00  00 

EE^mE  e  rmTTi,j{ri,r2) 

i=0  j=0  ri=0r2=0 

+  0(1  -qi  -  q2). 

Analogously  to  the  single  color  case, 

7rij(fe,;;x,t;  A)  =  Vk{ai{T  -  t))P;(a2(r  - 
The  equation  for  q  is 


(T-t)(6-l) 


log  +1 

\  J 


+  UmiT  -t)  -0  =  0, 
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so  that 


.  .  ,  _  Mm  _ Z _  1 

qm{x,t]A)  =  ttme  b-ie(x-t)(b-i)e  ^ 
For  i  =  0, . . .  ,1,1  +  l,j  =  0, . . . ,  J,  J  +  1  and  m  =  1,2  let 


Gi,j{x,t-,A)  =  I  1  -  A)  I  , 

\  fc=0  z=o  / 

(I+l  J+l  oo  oo 

qm{x,t-,A){T  -  t)  -  Xjj  rmTrijjk,  1]  X,  t;  A) 

i=0  j=0  fc=0  (=0 

G3{x,t-,A)  =  {1  -  qi{x,t-,A)  -  q2{x,t-,A)). 


When  discussing  uniqueness  of  the  multipliers  we  must  work  with  a  matrix  indexed 
by  the  subscripts  of  these  functions,  and  the  particular  ordering  of  the  i,j  as  sub¬ 
scripts  is  unimportant. 

We  next  present  three  lemmas  that  are  analogues  of  ones  proved  in  the  case  of 
a  single  color.  Since  the  proofs  of  the  first  two  are  also  direct  analogues  they  are 
omitted. 


Lemma  3.2  (General  properties).  For  any  {x,t)  G  V,  F{x,t;T)  is  nonempty, 
minimizing  measures  tt*  exist,  and  if  Xij  >  0  then  7r*j{k,l)  >  0  for  all  k  and  1. 

Lemma  3.3  (Characterization  of  the  minimizer).  For  any  {x,t)  G  V  there 
exists  A  G  M(-''+2)x(‘^+2)+3  so  that  G{x,t;A)  =  0,  and  7rij{k,l;x,t',  A),  qm{x,t-,A)  is 
a  minimizer  in  the  definition  ofU{x,t). 

Lemma  3.4  (Uniqueness  of  characterization).  For  {x,t)  G  V,  there  is  only  one 
A  G  mM+2)x(.^+2)+3  G{x,t,A)  =  0. 

Proof.  We  have 

=  7rij{k,l;x,t;A) 

=  k7Tij{k,l‘,x,t;  A) 

=  l7rij{k,k,x,t;A) 

=  -bh^rn{x,t-,A) 

and  all  other  partial  derivatives  are  zero.  As  in  the  single  color  case  it  is  enough 
to  show  the  negative  definiteness  of  D\G.  Using  a  suitable  dominating  function  to 


qm{x,t-,A), 


diTij 

d\ij 

diTij  {k,l;x,t;A 
dUl 

diTij  {k,l-,x,t;A 
dU2 

dqm(x,t;A) 

dqm{x,t\A) 

ae 
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justify  the  interchange  of  differentiation  and  summation  and  the  definitions 

CXD 

ttij  =  7Tij{k,l;x,t]A) 


rmTTij{ri,r2-,x,t-,A) 

ri,r2=0 

7+1  J+1  oo 

Ci,i  =  rlTTij{ri,r2;x,t-,A) 

i=0  j=0  ri,r2=0 

/+!  J+1  oo 

(^2,2  =  I]  r|7rij(ri,r2;x,t;  A) 

2=0  j=0  ri,r2=0 

7+1  J+1  oo 

^h2  =  e  nr27rij(ri,r2;x,t;  A), 

i=0  j=0  ri,r2=0 


DaG  equals  [with  qm  =  qm{x,  t;  A)] 


^  — «0,0 

0 

1 

•  + 

0 

r^0,0 

“-^2 

0  \ 

0 

—  Q!/+1,J+1 

_ j^7+l  j+l 

_ ^7'+l,j+l 

0 

r7u0,0 

-^0,0^1 

T-iJ+liJ+l 

•  •  •  -xi+i,j+iT^ 

—  C'1,2 

-5^91 

r7u0,0 

Xo,0-^  2 

rpJ+l,J+l 

•  •  •  -J;/+1,J+1?2 

-C'1,2 

-^g2-C2,2 

-b^^2 

V  0 

0 

91+92 

Since  diagonalizing  all  save  the  lower  right  3  x  3-submatrix  produces  strictly  negative 
values  on  the  diagonal,  we  need  only  check  the  negative  definiteness  of 


T.i.j  -  i+li  -  Ci,i  Y.i.i  -  Cl, 2  -5+91 

E«  -  Cl.2  El.,  -  Iff®  -  C2,2  -5+® 

“  (T-tUbl] 


Since 

(  -^91  0  -j^qi  \ 

0  -6^92 

\  F^'i'2  “  / 

is  obviously  negative  definite,  we  need  only  check  the  2x2  matrix 

(  E,,  -  C'1,2  \ 

E,,  ^E*"r2"  -  Cg2  E,,  J2"?  -  C'2.2  )  • 

However,  letting  7rij(ri,r2)  =  7rij{ri,r2]  x,t;  A)  and  pre-  and  post-multiplying  by 
the  nonzero  vector  (2;i,2;2)  produces 

Ev^fE  {zm  +  Z2r2)TTi,j{ri,r2)  -  E(^Hi  +  2^2^2)^715+  +  , r2)  •  E  7’2)^  <  0 
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Thus  the  entire  matrix  is  negative  definite. 


□ 


Since  we  have  restricted  to  the  case  of  MB  there  is  no  analogue  of  the  non¬ 
smooth  fnnction  r(x,f),  and  hence  the  existence  of  a  smooth  extension  of  [/  to  a 
neighborhood  of  "D  follows  directly  from  the  implicit  function  theorem.  The  next 
result  expresses  the  derivatives  in  terms  of  the  multipliers. 

Theorem  3.5.  Fix  {x,t)  G  D,  and  let  A*  he  the  associated  Lagrange  multiplier. 
Then 

D^^.U{x,t)  =  {\F-l) 

and 

DtU {x,  t)  =  {b-i)- 

Proof.  With 

H{x,t]A)  =  L{x,t]A,7Tij{-,x,t;A),qm{x,t]A)) 

we  can  write 

U{x,t)  =  H{x,t',  A{x,t)), 

where  A{x,  t)  is  the  nnique  solntion  to  the  constraint  equations.  Then 

D,,.U{x,t)  =  D,,.H{x,t-,A*)  +  DAH{x,t-,A*)D,,.A{x,t). 

As  in  the  single  color  case  DAH{x,t-,  A*)  =  0,  and  so  DxijU{x,t)  =  t;  A*), 

and  an  analogons  argument  also  gives  DtU{x,t)  =  DtF[{x,t;  A*).  Note  that  both 
7rij{k,  1;  X,  t]  A)  and  qm{x,  t;  A)  are  actually  independent  of  x,  and  hence  can  be  writ¬ 
ten  TTij{k,l;t;  A)  and  qmit]A).  Keeping  in  mind  the  t  dependence  bnt  temporarily 
suppressing  both  t  and  A*  in  the  notation,  we  will  nse 

(T  -t){b-  l)Riq  ||a)  =  -(T  -  t)  [qi^l  +  92^2]  +  [0*  -  {T  -  t){b  -  l)](gi  +  92)- 


The  qnantity  to  be  differentiated  is  thus 


7+1  J+1  00  00 

[K,j  -  l  +  kiil+  +2]  1)-{T  -  t)  [qiixl  -h  92+] 

i=0  j=0  k=0  1=0 

+  r-{T-t){b-l)]{qi+q2) 

7+1  J+1  /  00  00  \ 

+ E  E ^  “  E  E  0 1 

i=0  j=0  \  k=0  1=0  / 

(7+1  J+1  00  00 

gm(T  E  E vri,j(ri,r2) 

i=0  j=0  ri=0r2=0 

+9*{l-qi-q2) 

7+1  J+1  00  00  7+1  J+1 

=  “EE  E  E  T^ijik,  1)-{T  -  t){h  -  l)(gi  -h  q2)  +  EE  K,j^i,j  +  ■ 

i=0  j=0  k=0  1=0  i=0  j=0 
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Since 


dTTi,j{k,l]t]K)  ^  dVkiai{T  -  t))Vi{a2{T  -  t)) 


gAij  l+kfii+lfi2  £i4.k,j+i  Qq  {T  —  tY 

kU\  Ft 

k  I  1 


it  follows  that 


dqm{t]k) 


{T-tY{h-l) 


Qmit]  A), 


D_c/(x,t)  =  (A:,-i) 


7  +  1  J+l  CXD  OO  r 

DtU{x,t)  =  1- 

i=0 j=0  k=0 1=0  - 


T-t  T-t 


-Ki,j{k,l) 


+{b  -  1)  {qi  +  ga)  -  +  ^2) 


=  -1  +  gi  +  ga  +  (6  -  1) 


{T-t) 


=  (b-l)- 


(T-t)- 


For  the  particular  rate  function  of  interest  here, 

6(7)  =  sup[(7,g) -6i?(g||o)] 


=  -6inf  --{j,q)+R{q\\a) 
=  61og  ^e"^ai  +  e"^oa^  . 


Hence  by  (3.1),  the  PDF  to  be  satisfied  by  U  takes  the  form 


114  — 6 log  j  'Y  ^  Y^  Xi^j\  ai 

+  ^  E  02]  =0. 

\i,j,j<J  i,j,j=J+£  ) 


Theorem  3.6.  U  satisfies  (3.2)  onV. 
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Proof.  By  Theorem  3.5,  this  will  be  true  if 


(6-1)- 


(T-t) 


6 log  I  ^  e  ^ 


Xij  I  ai 


For  i  <  I 


+  j  ^  e  ^  Xij  j  02  I  =  0. 


TTi+ij{k,l)  _  ^i(A:  +  l) 


(3.3) 


TTij{k  +  1,0 

Summing  on  k  and  I  gives 

gAij— Ai+ij-  _  ® 


ai(r  - 1) 


,.,(rry  E 

and  the  analogous  formula 

e+o-+,i+i  =  ^  r2TTi,j{ri,r2) 

a2[l  —  t)  ^ 


ri,r2 


applies  for  j  <  J.  We  also  have 

g-M2 


1  = 


02  (T  - 1) 


^  ?'2vr/+ij(ri,r2) 


3  M2 


^1,^2 


02  (r  - 1) 


^  r27rj,j+i(ri,r2) 


ri,r2 


if  J  <  J  +  1  or  z 


1.  Hence 


i,j,i<I 


^i,j  =  ^  ^1— , 

Oi 


yy  e  j+1  +  yy 


Xjj  =  e  ^2 


-ui92 


Now  gi  +  (72  =  1  implies 


02 


0  =  log(gi(t;  A)  +  92(0  A)) 

Ml  9 _ M2 


log 

log 


oie  i>-i  e  +026  i>~i  e 

e 


_  Ml  _  M2 

Oie  *>-l  +  026  fc-l  1  + 


(T- 0(6-1) 


-  1. 
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The  left  hand  side  of  (3.3)  becomes 


(b  - -  blog  {  (e 

=  (fe-1)- 


oi  +  ^2 


<32 


9* 


=  -(6-1)  + 
=  -b  + 


{T-t) 
9* 


II  *  /i* 

If  *  9  1  \  o  j  ^  H-2  1 

—  61og  I  e~^ie  b-i  e(T-t)(b-i)  j  _j_  I  e“A‘2e  f>-i  e  (3’-0(<>-i) 


m2  _ _ e* 


<32 


(T-f) 

60* 


-6 


6Mi  \  b 


log  e  ^-1  ai  +  {  e  a2  + 


bfi2  \  b 


9* 


b{T-t){b-l)  6 


b9* 


{T-t){b-l)  {T-t){b-l) 


+  6 


=  0, 

and  the  theorem  is  proved. 


□ 


3.4  Minimizing  Trajectories 

We  end  this  section  by  stating  without  proof  the  form  of  the  minimizing  trajectories. 
As  in  the  case  of  a  single  color  we  consider  only  the  empty  initial  condition.  In 
contrast  with  that  case,  here  the  minimizing  q  must  be  determined  first  via  Lagrange 
multipliers.  Once  q  is  given,  we  define 

+o,o(ii,  ^2) 

/  J 

=  CVo{pqiti)VQ{pq2t2)  +  EE  {yk,l  -  CVk{pqiT)Vi{pq2T)) 

k=0  1=0 

and 

^  •  i-hYi-hy  , 

^i,j(.ti,t2)  =  - 7i— +0,0  (^1>^2). 

1.  J . 

In  terms  of  these  functions  we  set 

and  for  i  <  I  and  j  <  J 

00  00  ( 

v’i+,j{t)  =  (pi,j+it)  =  fijit),  +7+j+(t)  =  Y  +»d(^)- 

1=3+1  j=J+l  i=I+lj=J+l 

With  q  determined  via  Lagrange  multipliers,  the  parameters  p  >  0  and  C  >  0  are 
chosen  so  that 

pT  -  Elo  E/=o(^  +  m{pqiT)V,{pq2T)  T  -  ZLo  E/=o(^  + 

1  -  E[=o  E/=o  V^ipqiT)V,{pq2T)  1  -  ZLo  E/=o  PPj 

and 

^  ^  1  “  Ei=o  Ej=o  ~  Ei=o  Ej=o(^ 

~  1  -  E-=o  E/=o  V^ipqiT)V,ipq2T)  ~  pT  -  Zl=o  E/=o(^  +  m{pqiT)Vj{pq2T) ' 
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The  proof  that  these  trajectories  achieve  the  minimal  cost  parallels  that  of  the  single 
color  case,  with  an  appropriate  modification  of  the  notion  of  completely  monotone 
that  is  suitable  for  functions  of  two  independent  variables. 
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